Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
2026-06-18T13:59:15Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-06-18T13:59:15Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260618 13:59:15.774598 28048 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu
/tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.100.62:43001
--webserver_interface=127.27.100.62
--webserver_port=0
--builtin_ntp_servers=127.27.100.20:36797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.100.62:43001,127.27.100.61:45967,127.27.100.60:42083
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260618 13:59:16.089756 28056 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 13:59:16.090137 28056 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 13:59:16.090278 28056 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260618 13:59:16.098886 28056 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 13:59:16.099009 28056 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260618 13:59:16.099135 28056 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 13:59:16.099229 28056 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260618 13:59:16.109552 28056 master_runner.cc:387] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.100.20:36797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.27.100.62:43001,127.27.100.61:45967,127.27.100.60:42083
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.100.62:43001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.100.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision faec34f53d4c9dd1e3a6dd2b7314d2fc7aadff6c
build type FASTDEBUG
built by None at 18 Jun 2026 13:43:31 UTC on bdcb31816ec0
build id 12454
ASAN enabled
I20260618 13:59:16.111425 28056 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 13:59:16.113464 28056 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260618 13:59:16.121775 28062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 13:59:16.121737 28061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 13:59:16.121829 28064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 13:59:16.122467 28056 server_base.cc:1061] running on GCE node
I20260618 13:59:16.123180 28056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 13:59:16.124442 28056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 13:59:16.125638 28056 hybrid_clock.cc:648] HybridClock initialized: now 1781791156125606 us; error 25 us; skew 500 ppm
I20260618 13:59:16.125924 28056 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 13:59:16.128211 28056 webserver.cc:533] Webserver started at http://127.27.100.62:35969/ using document root <none> and password file <none>
I20260618 13:59:16.128857 28056 fs_manager.cc:362] Metadata directory not provided
I20260618 13:59:16.128942 28056 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 13:59:16.129231 28056 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 13:59:16.131505 28056 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-0/data/instance:
uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff"
format_stamp: "Formatted at 2026-06-18 13:59:16 on dist-test-slave-bmq2"
I20260618 13:59:16.132215 28056 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-0/wal/instance:
uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff"
format_stamp: "Formatted at 2026-06-18 13:59:16 on dist-test-slave-bmq2"
I20260618 13:59:16.137075 28056 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260618 13:59:16.140249 28070 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 13:59:16.141458 28056 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260618 13:59:16.141585 28056 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-0/data,/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-0/wal
uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff"
format_stamp: "Formatted at 2026-06-18 13:59:16 on dist-test-slave-bmq2"
I20260618 13:59:16.141801 28056 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 13:59:16.166221 28056 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260618 13:59:16.167153 28056 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 13:59:16.167385 28056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 13:59:16.187098 28056 rpc_server.cc:307] RPC server started. Bound to: 127.27.100.62:43001
I20260618 13:59:16.187134 28121 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.100.62:43001 every 8 connection(s)
I20260618 13:59:16.188606 28056 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-0/data/info.pb
I20260618 13:59:16.192672 28122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:16.198109 28122 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } has no permanent_uuid. Determining permanent_uuid...
I20260618 13:59:16.199559 28048 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu as pid 28056
I20260618 13:59:16.199863 28048 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-0/wal/instance
I20260618 13:59:16.202807 28048 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu
/tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.100.61:45967
--webserver_interface=127.27.100.61
--webserver_port=0
--builtin_ntp_servers=127.27.100.20:36797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.100.62:43001,127.27.100.61:45967,127.27.100.60:42083
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260618 13:59:16.217978 28122 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } has no permanent_uuid. Determining permanent_uuid...
W20260618 13:59:16.219213 28074 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.100.61:45967: connect: Connection refused (error 111)
W20260618 13:59:16.220800 28122 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.100.61:45967: Network error: Client connection negotiation failed: client connection to 127.27.100.61:45967: connect: Connection refused (error 111)
I20260618 13:59:16.270191 28122 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } attempt: 1
W20260618 13:59:16.272239 28122 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.100.61:45967: Network error: Client connection negotiation failed: client connection to 127.27.100.61:45967: connect: Connection refused (error 111)
I20260618 13:59:16.340567 28122 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } attempt: 2
W20260618 13:59:16.342703 28122 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.100.61:45967: Network error: Client connection negotiation failed: client connection to 127.27.100.61:45967: connect: Connection refused (error 111)
I20260618 13:59:16.433974 28122 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } attempt: 3
W20260618 13:59:16.436194 28122 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.100.61:45967: Network error: Client connection negotiation failed: client connection to 127.27.100.61:45967: connect: Connection refused (error 111)
W20260618 13:59:16.522308 28126 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 13:59:16.522671 28126 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 13:59:16.522753 28126 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260618 13:59:16.531038 28126 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 13:59:16.531118 28126 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260618 13:59:16.531170 28126 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 13:59:16.531212 28126 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260618 13:59:16.540940 28126 master_runner.cc:387] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.100.20:36797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.27.100.62:43001,127.27.100.61:45967,127.27.100.60:42083
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.100.61:45967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.100.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision faec34f53d4c9dd1e3a6dd2b7314d2fc7aadff6c
build type FASTDEBUG
built by None at 18 Jun 2026 13:43:31 UTC on bdcb31816ec0
build id 12454
ASAN enabled
I20260618 13:59:16.542440 28126 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 13:59:16.544135 28126 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260618 13:59:16.552273 28132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 13:59:16.552322 28133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 13:59:16.552809 28135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 13:59:16.553490 28126 server_base.cc:1061] running on GCE node
I20260618 13:59:16.554210 28126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 13:59:16.555560 28126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 13:59:16.556802 28126 hybrid_clock.cc:648] HybridClock initialized: now 1781791156556773 us; error 44 us; skew 500 ppm
I20260618 13:59:16.557169 28126 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 13:59:16.559242 28126 webserver.cc:533] Webserver started at http://127.27.100.61:35663/ using document root <none> and password file <none>
I20260618 13:59:16.559938 28126 fs_manager.cc:362] Metadata directory not provided
I20260618 13:59:16.560098 28126 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 13:59:16.560436 28126 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 13:59:16.562795 28126 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/data/instance:
uuid: "a950ab7cd66b4cb0b29592aa692c2dd0"
format_stamp: "Formatted at 2026-06-18 13:59:16 on dist-test-slave-bmq2"
I20260618 13:59:16.563553 28126 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/wal/instance:
uuid: "a950ab7cd66b4cb0b29592aa692c2dd0"
format_stamp: "Formatted at 2026-06-18 13:59:16 on dist-test-slave-bmq2"
I20260618 13:59:16.568497 28126 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 13:59:16.571686 28141 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 13:59:16.572928 28126 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 13:59:16.573108 28126 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/data,/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/wal
uuid: "a950ab7cd66b4cb0b29592aa692c2dd0"
format_stamp: "Formatted at 2026-06-18 13:59:16 on dist-test-slave-bmq2"
I20260618 13:59:16.573311 28126 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 13:59:16.579459 28122 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } attempt: 4
W20260618 13:59:16.581193 28122 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.100.61:45967: Network error: Client connection negotiation failed: client connection to 127.27.100.61:45967: connect: Connection refused (error 111)
I20260618 13:59:16.600296 28126 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260618 13:59:16.601169 28126 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 13:59:16.601492 28126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 13:59:16.621021 28126 rpc_server.cc:307] RPC server started. Bound to: 127.27.100.61:45967
I20260618 13:59:16.621032 28192 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.100.61:45967 every 8 connection(s)
I20260618 13:59:16.622567 28126 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/data/info.pb
I20260618 13:59:16.626677 28193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:16.629127 28048 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu as pid 28126
I20260618 13:59:16.629331 28048 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/wal/instance
I20260618 13:59:16.630749 28048 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu
/tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.100.60:42083
--webserver_interface=127.27.100.60
--webserver_port=0
--builtin_ntp_servers=127.27.100.20:36797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.100.62:43001,127.27.100.61:45967,127.27.100.60:42083
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260618 13:59:16.634218 28193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } has no permanent_uuid. Determining permanent_uuid...
I20260618 13:59:16.647719 28193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } has no permanent_uuid. Determining permanent_uuid...
I20260618 13:59:16.656893 28193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } has no permanent_uuid. Determining permanent_uuid...
W20260618 13:59:16.658066 28142 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111)
W20260618 13:59:16.659749 28193 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.100.60:42083: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111)
I20260618 13:59:16.709147 28193 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } attempt: 1
W20260618 13:59:16.711202 28193 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.100.60:42083: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111)
I20260618 13:59:16.779454 28193 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } attempt: 2
W20260618 13:59:16.781422 28193 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.100.60:42083: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111)
I20260618 13:59:16.872696 28193 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } attempt: 3
W20260618 13:59:16.874766 28193 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.100.60:42083: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111)
I20260618 13:59:16.880443 28122 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } attempt: 5
I20260618 13:59:16.887301 28122 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } has no permanent_uuid. Determining permanent_uuid...
W20260618 13:59:16.889199 28122 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.100.60:42083: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111)
I20260618 13:59:16.940446 28122 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } attempt: 1
W20260618 13:59:16.942688 28122 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.100.60:42083: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111)
W20260618 13:59:16.952893 28195 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 13:59:16.953189 28195 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 13:59:16.953334 28195 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260618 13:59:16.961743 28195 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 13:59:16.961823 28195 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260618 13:59:16.961917 28195 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 13:59:16.961980 28195 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260618 13:59:16.972095 28195 master_runner.cc:387] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.100.20:36797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.27.100.62:43001,127.27.100.61:45967,127.27.100.60:42083
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.100.60:42083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.100.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision faec34f53d4c9dd1e3a6dd2b7314d2fc7aadff6c
build type FASTDEBUG
built by None at 18 Jun 2026 13:43:31 UTC on bdcb31816ec0
build id 12454
ASAN enabled
I20260618 13:59:16.973680 28195 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 13:59:16.975536 28195 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260618 13:59:16.983420 28204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 13:59:16.983484 28203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 13:59:16.984170 28206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 13:59:16.985350 28195 server_base.cc:1061] running on GCE node
I20260618 13:59:16.986042 28195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 13:59:16.987468 28195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 13:59:16.988675 28195 hybrid_clock.cc:648] HybridClock initialized: now 1781791156988638 us; error 28 us; skew 500 ppm
I20260618 13:59:16.988979 28195 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 13:59:16.991158 28195 webserver.cc:533] Webserver started at http://127.27.100.60:38293/ using document root <none> and password file <none>
I20260618 13:59:16.991865 28195 fs_manager.cc:362] Metadata directory not provided
I20260618 13:59:16.991951 28195 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 13:59:16.992240 28195 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 13:59:16.994546 28195 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data/instance:
uuid: "7637285570574a6dbcef1de6f84ecac0"
format_stamp: "Formatted at 2026-06-18 13:59:16 on dist-test-slave-bmq2"
I20260618 13:59:16.995277 28195 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/wal/instance:
uuid: "7637285570574a6dbcef1de6f84ecac0"
format_stamp: "Formatted at 2026-06-18 13:59:16 on dist-test-slave-bmq2"
I20260618 13:59:17.000281 28195 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 13:59:17.003361 28212 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 13:59:17.004602 28195 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 13:59:17.004725 28195 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data,/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/wal
uuid: "7637285570574a6dbcef1de6f84ecac0"
format_stamp: "Formatted at 2026-06-18 13:59:16 on dist-test-slave-bmq2"
I20260618 13:59:17.004881 28195 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 13:59:17.011051 28122 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } attempt: 2
W20260618 13:59:17.012902 28122 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.100.60:42083: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111)
I20260618 13:59:17.018005 28193 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } attempt: 4
W20260618 13:59:17.019673 28193 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.100.60:42083: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111)
I20260618 13:59:17.024260 28195 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260618 13:59:17.025028 28195 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 13:59:17.025246 28195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 13:59:17.044771 28195 rpc_server.cc:307] RPC server started. Bound to: 127.27.100.60:42083
I20260618 13:59:17.044791 28263 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.100.60:42083 every 8 connection(s)
I20260618 13:59:17.046298 28195 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data/info.pb
I20260618 13:59:17.048177 28048 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu as pid 28195
I20260618 13:59:17.048363 28048 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/wal/instance
I20260618 13:59:17.051474 28264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:17.058102 28264 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } has no permanent_uuid. Determining permanent_uuid...
I20260618 13:59:17.070535 28264 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } has no permanent_uuid. Determining permanent_uuid...
I20260618 13:59:17.079800 28264 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } has no permanent_uuid. Determining permanent_uuid...
I20260618 13:59:17.090931 28264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Bootstrap starting.
I20260618 13:59:17.094203 28264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:17.095216 28264 log.cc:826] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Log is configured to *not* fsync() on all Append() calls
I20260618 13:59:17.097503 28264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: No bootstrap required, opened a new log
I20260618 13:59:17.102557 28264 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:17.102803 28264 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:17.102880 28264 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7637285570574a6dbcef1de6f84ecac0, State: Initialized, Role: FOLLOWER
I20260618 13:59:17.103384 28264 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:17.104171 28273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:17.104440 28273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 13:59:17.105123 28264 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: configured and running, proceeding with master startup.
W20260618 13:59:17.114982 28284 catalog_manager.cc:1593] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 13:59:17.115128 28284 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 13:59:17.112576 28195 master_runner.cc:187] Error getting master registration for 127.27.100.62:43001: OK, instance_id { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" instance_seqno: 1781791156181707 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 13:59:17.119118 28122 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } attempt: 3
I20260618 13:59:17.119194 28195 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260618 13:59:17.131349 28122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff: Bootstrap starting.
I20260618 13:59:17.135353 28122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:17.136747 28122 log.cc:826] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff: Log is configured to *not* fsync() on all Append() calls
I20260618 13:59:17.139638 28122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff: No bootstrap required, opened a new log
I20260618 13:59:17.146870 28122 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:17.147168 28122 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:17.147296 28122 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4d4f4566be42dfbafb7c0e40fb17ff, State: Initialized, Role: FOLLOWER
I20260618 13:59:17.147960 28122 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:17.148854 28287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:17.149114 28287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [sys.catalog]: This master's current role is: FOLLOWER
I20260618 13:59:17.150175 28122 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [sys.catalog]: configured and running, proceeding with master startup.
W20260618 13:59:17.160979 28298 catalog_manager.cc:1593] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 13:59:17.161170 28298 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 13:59:17.159597 28056 master_runner.cc:187] Error getting master registration for 127.27.100.61:45967: OK, instance_id { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" instance_seqno: 1781791156615745 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 13:59:17.165814 28056 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260618 13:59:17.169858 28076 tablet.cc:2406] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 13:59:17.277281 28218 tablet.cc:2406] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 13:59:17.318883 28193 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } attempt: 5
I20260618 13:59:17.331753 28193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Bootstrap starting.
I20260618 13:59:17.335916 28193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:17.337312 28193 log.cc:826] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Log is configured to *not* fsync() on all Append() calls
I20260618 13:59:17.340474 28193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: No bootstrap required, opened a new log
I20260618 13:59:17.348150 28193 raft_consensus.cc:359] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:17.348563 28193 raft_consensus.cc:385] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:17.348719 28193 raft_consensus.cc:740] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a950ab7cd66b4cb0b29592aa692c2dd0, State: Initialized, Role: FOLLOWER
I20260618 13:59:17.349480 28193 consensus_queue.cc:260] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:17.350544 28301 sys_catalog.cc:455] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:17.350804 28301 sys_catalog.cc:458] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 13:59:17.351956 28193 sys_catalog.cc:565] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: configured and running, proceeding with master startup.
W20260618 13:59:17.353947 28147 tablet.cc:2406] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 13:59:17.365912 28312 catalog_manager.cc:1593] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 13:59:17.366086 28312 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 13:59:17.370349 28126 master_runner.cc:264] No leader master found from master a950ab7cd66b4cb0b29592aa692c2dd0
I20260618 13:59:17.370549 28126 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260618 13:59:17.464004 28273 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:17.464290 28273 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:17.466130 28273 leader_election.cc:290] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4d4f4566be42dfbafb7c0e40fb17ff (127.27.100.62:43001), a950ab7cd66b4cb0b29592aa692c2dd0 (127.27.100.61:45967)
I20260618 13:59:17.467326 28168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7637285570574a6dbcef1de6f84ecac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" is_pre_election: true
I20260618 13:59:17.467900 28168 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7637285570574a6dbcef1de6f84ecac0 in term 0.
I20260618 13:59:17.468686 28216 leader_election.cc:304] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7637285570574a6dbcef1de6f84ecac0, a950ab7cd66b4cb0b29592aa692c2dd0; no voters:
I20260618 13:59:17.469883 28097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7637285570574a6dbcef1de6f84ecac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" is_pre_election: true
I20260618 13:59:17.470358 28097 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7637285570574a6dbcef1de6f84ecac0 in term 0.
I20260618 13:59:17.471318 28273 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:17.471480 28273 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:17.471596 28273 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:17.474925 28273 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:17.476056 28097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7637285570574a6dbcef1de6f84ecac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff"
I20260618 13:59:17.476243 28097 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:17.479774 28097 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7637285570574a6dbcef1de6f84ecac0 in term 1.
I20260618 13:59:17.480288 28215 leader_election.cc:304] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7637285570574a6dbcef1de6f84ecac0, 9e4d4f4566be42dfbafb7c0e40fb17ff; no voters:
I20260618 13:59:17.482297 28168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7637285570574a6dbcef1de6f84ecac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0"
I20260618 13:59:17.482542 28168 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:17.485791 28168 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7637285570574a6dbcef1de6f84ecac0 in term 1.
I20260618 13:59:17.486596 28273 leader_election.cc:290] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [CANDIDATE]: Term 1 election: Requested vote from peers 9e4d4f4566be42dfbafb7c0e40fb17ff (127.27.100.62:43001), a950ab7cd66b4cb0b29592aa692c2dd0 (127.27.100.61:45967)
I20260618 13:59:17.487041 28273 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:17.487504 28273 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 1 LEADER]: Becoming Leader. State: Replica: 7637285570574a6dbcef1de6f84ecac0, State: Running, Role: LEADER
I20260618 13:59:17.488237 28273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:17.505275 28314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7637285570574a6dbcef1de6f84ecac0. Latest consensus state: current_term: 1 leader_uuid: "7637285570574a6dbcef1de6f84ecac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:17.505728 28314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: This master's current role is: LEADER
I20260618 13:59:17.510756 28316 catalog_manager.cc:1510] Loading table and tablet metadata into memory...
I20260618 13:59:17.513203 28316 catalog_manager.cc:1519] Initializing Kudu cluster ID...
I20260618 13:59:17.528303 28168 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 1 FOLLOWER]: Refusing update from remote peer 7637285570574a6dbcef1de6f84ecac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 13:59:17.529467 28314 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:17.540022 28097 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 1 FOLLOWER]: Refusing update from remote peer 7637285570574a6dbcef1de6f84ecac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 13:59:17.541085 28314 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:17.573799 28287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7637285570574a6dbcef1de6f84ecac0. Latest consensus state: current_term: 1 leader_uuid: "7637285570574a6dbcef1de6f84ecac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:17.574018 28287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [sys.catalog]: This master's current role is: FOLLOWER
I20260618 13:59:17.576540 28317 mvcc.cc:204] Tried to move back new op lower bound from 7298216581224714240 to 7298216581083992064. Current Snapshot: MvccSnapshot[applied={T|T < 7298216581224714240}]
I20260618 13:59:17.582396 28319 mvcc.cc:204] Tried to move back new op lower bound from 7298216581224714240 to 7298216581083992064. Current Snapshot: MvccSnapshot[applied={T|T < 7298216581224714240}]
I20260618 13:59:17.582906 28287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7637285570574a6dbcef1de6f84ecac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:17.583132 28287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [sys.catalog]: This master's current role is: FOLLOWER
I20260618 13:59:17.588949 28273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7637285570574a6dbcef1de6f84ecac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:17.589196 28273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: This master's current role is: LEADER
I20260618 13:59:17.591670 28316 catalog_manager.cc:1382] Generated new cluster ID: 7094aaeda1bc41d1ac58a313dfa3bf98
I20260618 13:59:17.591760 28316 catalog_manager.cc:1530] Initializing Kudu internal certificate authority...
I20260618 13:59:17.612331 28316 catalog_manager.cc:1405] Generated new certificate authority record
I20260618 13:59:17.613963 28316 catalog_manager.cc:1539] Loading token signing keys...
I20260618 13:59:17.614286 28301 sys_catalog.cc:455] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7637285570574a6dbcef1de6f84ecac0. Latest consensus state: current_term: 1 leader_uuid: "7637285570574a6dbcef1de6f84ecac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:17.614550 28301 sys_catalog.cc:458] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 13:59:17.615737 28314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7637285570574a6dbcef1de6f84ecac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:17.616011 28314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: This master's current role is: LEADER
I20260618 13:59:17.630656 28301 sys_catalog.cc:455] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7637285570574a6dbcef1de6f84ecac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:17.630873 28301 sys_catalog.cc:458] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 13:59:17.632148 28316 catalog_manager.cc:6069] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Generated new TSK 0
I20260618 13:59:17.633119 28316 catalog_manager.cc:1549] Initializing in-progress tserver states...
I20260618 13:59:17.645718 28048 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu
/tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.100.1:0
--local_ip_for_outbound_sockets=127.27.100.1
--webserver_interface=127.27.100.1
--webserver_port=0
--tserver_master_addrs=127.27.100.62:43001,127.27.100.61:45967,127.27.100.60:42083
--builtin_ntp_servers=127.27.100.20:36797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20260618 13:59:17.980540 28329 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 13:59:17.980836 28329 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 13:59:17.980978 28329 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260618 13:59:17.989771 28329 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260618 13:59:17.990005 28329 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.100.1
I20260618 13:59:18.000180 28329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.100.20:36797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.100.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.100.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.27.100.62:43001,127.27.100.61:45967,127.27.100.60:42083
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.100.1
--log_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision faec34f53d4c9dd1e3a6dd2b7314d2fc7aadff6c
build type FASTDEBUG
built by None at 18 Jun 2026 13:43:31 UTC on bdcb31816ec0
build id 12454
ASAN enabled
I20260618 13:59:18.001864 28329 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 13:59:18.003819 28329 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260618 13:59:18.013103 28334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 13:59:18.013135 28335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 13:59:18.013842 28337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 13:59:18.014482 28329 server_base.cc:1061] running on GCE node
I20260618 13:59:18.015084 28329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 13:59:18.015991 28329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 13:59:18.017230 28329 hybrid_clock.cc:648] HybridClock initialized: now 1781791158017191 us; error 53 us; skew 500 ppm
I20260618 13:59:18.017529 28329 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 13:59:18.019634 28329 webserver.cc:533] Webserver started at http://127.27.100.1:46603/ using document root <none> and password file <none>
I20260618 13:59:18.020367 28329 fs_manager.cc:362] Metadata directory not provided
I20260618 13:59:18.020500 28329 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 13:59:18.020851 28329 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 13:59:18.023237 28329 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/data/instance:
uuid: "6fc808af0e884ed8836561665d8e0e7b"
format_stamp: "Formatted at 2026-06-18 13:59:18 on dist-test-slave-bmq2"
I20260618 13:59:18.023993 28329 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/instance:
uuid: "6fc808af0e884ed8836561665d8e0e7b"
format_stamp: "Formatted at 2026-06-18 13:59:18 on dist-test-slave-bmq2"
I20260618 13:59:18.029426 28329 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 13:59:18.032684 28343 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 13:59:18.033993 28329 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260618 13:59:18.034173 28329 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal
uuid: "6fc808af0e884ed8836561665d8e0e7b"
format_stamp: "Formatted at 2026-06-18 13:59:18 on dist-test-slave-bmq2"
I20260618 13:59:18.034394 28329 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 13:59:18.069857 28329 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260618 13:59:18.070842 28329 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 13:59:18.071120 28329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 13:59:18.072053 28329 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260618 13:59:18.073694 28329 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 13:59:18.073815 28329 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 13:59:18.073941 28329 ts_tablet_manager.cc:616] Registered 0 tablets
I20260618 13:59:18.074029 28329 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 13:59:18.108052 28329 rpc_server.cc:307] RPC server started. Bound to: 127.27.100.1:42675
I20260618 13:59:18.108141 28455 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.100.1:42675 every 8 connection(s)
I20260618 13:59:18.109589 28329 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/data/info.pb
I20260618 13:59:18.116786 28048 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu as pid 28329
I20260618 13:59:18.117096 28048 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/instance
I20260618 13:59:18.125238 28048 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu
/tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.100.2:0
--local_ip_for_outbound_sockets=127.27.100.2
--webserver_interface=127.27.100.2
--webserver_port=0
--tserver_master_addrs=127.27.100.62:43001,127.27.100.61:45967,127.27.100.60:42083
--builtin_ntp_servers=127.27.100.20:36797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260618 13:59:18.128207 28457 heartbeater.cc:344] Connected to a master server at 127.27.100.60:42083
I20260618 13:59:18.128696 28457 heartbeater.cc:461] Registering TS with master...
I20260618 13:59:18.130028 28457 heartbeater.cc:507] Master 127.27.100.60:42083 requested a full tablet report, sending...
I20260618 13:59:18.133097 28229 ts_manager.cc:194] Registered new tserver with Master: 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:18.133483 28466 raft_consensus.cc:493] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7637285570574a6dbcef1de6f84ecac0)
I20260618 13:59:18.133678 28466 raft_consensus.cc:515] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:18.135242 28229 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.27.100.1:59103
I20260618 13:59:18.136488 28458 heartbeater.cc:344] Connected to a master server at 127.27.100.61:45967
I20260618 13:59:18.136680 28458 heartbeater.cc:461] Registering TS with master...
I20260618 13:59:18.136817 28096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" is_pre_election: true
I20260618 13:59:18.137230 28458 heartbeater.cc:507] Master 127.27.100.61:45967 requested a full tablet report, sending...
I20260618 13:59:18.137964 28466 leader_election.cc:290] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e4d4f4566be42dfbafb7c0e40fb17ff (127.27.100.62:43001), 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083)
I20260618 13:59:18.138321 28239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7637285570574a6dbcef1de6f84ecac0" is_pre_election: true
I20260618 13:59:18.139178 28142 leader_election.cc:304] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a950ab7cd66b4cb0b29592aa692c2dd0; no voters: 7637285570574a6dbcef1de6f84ecac0, 9e4d4f4566be42dfbafb7c0e40fb17ff
I20260618 13:59:18.139660 28456 heartbeater.cc:344] Connected to a master server at 127.27.100.62:43001
I20260618 13:59:18.139801 28456 heartbeater.cc:461] Registering TS with master...
I20260618 13:59:18.139930 28466 raft_consensus.cc:2749] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:18.140285 28456 heartbeater.cc:507] Master 127.27.100.62:43001 requested a full tablet report, sending...
I20260618 13:59:18.141994 28087 ts_manager.cc:194] Registered new tserver with Master: 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:18.142867 28158 ts_manager.cc:194] Registered new tserver with Master: 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:18.163023 28298 catalog_manager.cc:1294] Loaded cluster ID: 7094aaeda1bc41d1ac58a313dfa3bf98
I20260618 13:59:18.163156 28298 catalog_manager.cc:1587] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff: loading cluster ID for follower catalog manager: success
I20260618 13:59:18.165575 28298 catalog_manager.cc:1609] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff: acquiring CA information for follower catalog manager: success
I20260618 13:59:18.167354 28298 catalog_manager.cc:1637] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 13:59:18.367973 28312 catalog_manager.cc:1294] Loaded cluster ID: 7094aaeda1bc41d1ac58a313dfa3bf98
I20260618 13:59:18.368101 28312 catalog_manager.cc:1587] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: loading cluster ID for follower catalog manager: success
I20260618 13:59:18.370707 28312 catalog_manager.cc:1609] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: acquiring CA information for follower catalog manager: success
I20260618 13:59:18.372376 28312 catalog_manager.cc:1637] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260618 13:59:18.458236 28467 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 13:59:18.458616 28467 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 13:59:18.458761 28467 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260618 13:59:18.466967 28467 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260618 13:59:18.467175 28467 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.100.2
I20260618 13:59:18.476742 28467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.100.20:36797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.100.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.100.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.27.100.62:43001,127.27.100.61:45967,127.27.100.60:42083
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.100.2
--log_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision faec34f53d4c9dd1e3a6dd2b7314d2fc7aadff6c
build type FASTDEBUG
built by None at 18 Jun 2026 13:43:31 UTC on bdcb31816ec0
build id 12454
ASAN enabled
I20260618 13:59:18.478575 28467 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 13:59:18.480404 28467 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260618 13:59:18.489544 28473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 13:59:18.489506 28472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 13:59:18.490564 28475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 13:59:18.490756 28467 server_base.cc:1061] running on GCE node
I20260618 13:59:18.491305 28467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 13:59:18.492151 28467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 13:59:18.493382 28467 hybrid_clock.cc:648] HybridClock initialized: now 1781791158493360 us; error 52 us; skew 500 ppm
I20260618 13:59:18.493678 28467 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 13:59:18.495704 28467 webserver.cc:533] Webserver started at http://127.27.100.2:32919/ using document root <none> and password file <none>
I20260618 13:59:18.496399 28467 fs_manager.cc:362] Metadata directory not provided
I20260618 13:59:18.496531 28467 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 13:59:18.496881 28467 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 13:59:18.499212 28467 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/data/instance:
uuid: "97859748af2c41879ba2a668dda999d2"
format_stamp: "Formatted at 2026-06-18 13:59:18 on dist-test-slave-bmq2"
I20260618 13:59:18.499936 28467 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/instance:
uuid: "97859748af2c41879ba2a668dda999d2"
format_stamp: "Formatted at 2026-06-18 13:59:18 on dist-test-slave-bmq2"
I20260618 13:59:18.504705 28467 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260618 13:59:18.507829 28481 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 13:59:18.509079 28467 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 13:59:18.509243 28467 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal
uuid: "97859748af2c41879ba2a668dda999d2"
format_stamp: "Formatted at 2026-06-18 13:59:18 on dist-test-slave-bmq2"
I20260618 13:59:18.509442 28467 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 13:59:18.525657 28467 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260618 13:59:18.526392 28467 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 13:59:18.526726 28467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 13:59:18.527597 28467 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260618 13:59:18.529101 28467 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 13:59:18.529218 28467 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 13:59:18.529347 28467 ts_tablet_manager.cc:616] Registered 0 tablets
I20260618 13:59:18.529436 28467 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 13:59:18.564172 28467 rpc_server.cc:307] RPC server started. Bound to: 127.27.100.2:40749
I20260618 13:59:18.564254 28593 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.100.2:40749 every 8 connection(s)
I20260618 13:59:18.565565 28467 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/data/info.pb
I20260618 13:59:18.574012 28048 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu as pid 28467
I20260618 13:59:18.574246 28048 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/instance
I20260618 13:59:18.581219 28048 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu
/tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.100.3:0
--local_ip_for_outbound_sockets=127.27.100.3
--webserver_interface=127.27.100.3
--webserver_port=0
--tserver_master_addrs=127.27.100.62:43001,127.27.100.61:45967,127.27.100.60:42083
--builtin_ntp_servers=127.27.100.20:36797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260618 13:59:18.590351 28594 heartbeater.cc:344] Connected to a master server at 127.27.100.62:43001
I20260618 13:59:18.590444 28595 heartbeater.cc:344] Connected to a master server at 127.27.100.60:42083
I20260618 13:59:18.590785 28596 heartbeater.cc:344] Connected to a master server at 127.27.100.61:45967
I20260618 13:59:18.590764 28594 heartbeater.cc:461] Registering TS with master...
I20260618 13:59:18.590770 28595 heartbeater.cc:461] Registering TS with master...
I20260618 13:59:18.590951 28596 heartbeater.cc:461] Registering TS with master...
I20260618 13:59:18.592108 28595 heartbeater.cc:507] Master 127.27.100.60:42083 requested a full tablet report, sending...
I20260618 13:59:18.592132 28596 heartbeater.cc:507] Master 127.27.100.61:45967 requested a full tablet report, sending...
I20260618 13:59:18.592373 28594 heartbeater.cc:507] Master 127.27.100.62:43001 requested a full tablet report, sending...
I20260618 13:59:18.593801 28229 ts_manager.cc:194] Registered new tserver with Master: 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:18.593999 28158 ts_manager.cc:194] Registered new tserver with Master: 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:18.594707 28229 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.27.100.2:59405
I20260618 13:59:18.594784 28087 ts_manager.cc:194] Registered new tserver with Master: 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:18.641672 28457 heartbeater.cc:499] Master 127.27.100.60:42083 was elected leader, sending a full tablet report...
W20260618 13:59:18.931088 28601 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 13:59:18.931373 28601 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 13:59:18.931464 28601 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260618 13:59:18.940594 28601 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260618 13:59:18.940814 28601 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.100.3
I20260618 13:59:18.951259 28601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.100.20:36797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.100.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.100.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.27.100.62:43001,127.27.100.61:45967,127.27.100.60:42083
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.100.3
--log_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision faec34f53d4c9dd1e3a6dd2b7314d2fc7aadff6c
build type FASTDEBUG
built by None at 18 Jun 2026 13:43:31 UTC on bdcb31816ec0
build id 12454
ASAN enabled
I20260618 13:59:18.952991 28601 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 13:59:18.954876 28601 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260618 13:59:18.963950 28607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 13:59:18.963871 28606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 13:59:18.964596 28609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 13:59:18.965695 28601 server_base.cc:1061] running on GCE node
I20260618 13:59:18.966254 28601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 13:59:18.967199 28601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 13:59:18.968436 28601 hybrid_clock.cc:648] HybridClock initialized: now 1781791158968398 us; error 50 us; skew 500 ppm
I20260618 13:59:18.968747 28601 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 13:59:18.970835 28601 webserver.cc:533] Webserver started at http://127.27.100.3:40625/ using document root <none> and password file <none>
I20260618 13:59:18.971572 28601 fs_manager.cc:362] Metadata directory not provided
I20260618 13:59:18.971704 28601 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 13:59:18.972033 28601 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 13:59:18.974367 28601 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/data/instance:
uuid: "d6a169b16ced46ca9cfbc631709bb257"
format_stamp: "Formatted at 2026-06-18 13:59:18 on dist-test-slave-bmq2"
I20260618 13:59:18.975208 28601 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/instance:
uuid: "d6a169b16ced46ca9cfbc631709bb257"
format_stamp: "Formatted at 2026-06-18 13:59:18 on dist-test-slave-bmq2"
I20260618 13:59:18.980672 28601 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 13:59:18.984118 28615 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 13:59:18.985641 28601 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:18.985795 28601 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal
uuid: "d6a169b16ced46ca9cfbc631709bb257"
format_stamp: "Formatted at 2026-06-18 13:59:18 on dist-test-slave-bmq2"
I20260618 13:59:18.985980 28601 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 13:59:19.004951 28601 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260618 13:59:19.005898 28601 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 13:59:19.006140 28601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 13:59:19.007130 28601 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260618 13:59:19.008956 28601 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 13:59:19.009105 28601 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 13:59:19.009194 28601 ts_tablet_manager.cc:616] Registered 0 tablets
I20260618 13:59:19.009265 28601 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 13:59:19.042579 28601 rpc_server.cc:307] RPC server started. Bound to: 127.27.100.3:35503
I20260618 13:59:19.042661 28727 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.100.3:35503 every 8 connection(s)
I20260618 13:59:19.044037 28601 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/data/info.pb
I20260618 13:59:19.052380 28048 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu as pid 28601
I20260618 13:59:19.052604 28048 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/instance
I20260618 13:59:19.069087 28730 heartbeater.cc:344] Connected to a master server at 127.27.100.61:45967
I20260618 13:59:19.069552 28730 heartbeater.cc:461] Registering TS with master...
I20260618 13:59:19.070672 28730 heartbeater.cc:507] Master 127.27.100.61:45967 requested a full tablet report, sending...
I20260618 13:59:19.072614 28158 ts_manager.cc:194] Registered new tserver with Master: d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:19.074095 28729 heartbeater.cc:344] Connected to a master server at 127.27.100.60:42083
I20260618 13:59:19.074259 28729 heartbeater.cc:461] Registering TS with master...
I20260618 13:59:19.074954 28729 heartbeater.cc:507] Master 127.27.100.60:42083 requested a full tablet report, sending...
I20260618 13:59:19.075170 28728 heartbeater.cc:344] Connected to a master server at 127.27.100.62:43001
I20260618 13:59:19.075284 28728 heartbeater.cc:461] Registering TS with master...
I20260618 13:59:19.075784 28728 heartbeater.cc:507] Master 127.27.100.62:43001 requested a full tablet report, sending...
I20260618 13:59:19.075981 28228 ts_manager.cc:194] Registered new tserver with Master: d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:19.076102 28735 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7637285570574a6dbcef1de6f84ecac0)
I20260618 13:59:19.076318 28735 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:19.077067 28228 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.27.100.3:55949
I20260618 13:59:19.077194 28087 ts_manager.cc:194] Registered new tserver with Master: d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:19.078262 28735 leader_election.cc:290] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a950ab7cd66b4cb0b29592aa692c2dd0 (127.27.100.61:45967), 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083)
I20260618 13:59:19.078274 28239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7637285570574a6dbcef1de6f84ecac0" is_pre_election: true
I20260618 13:59:19.078280 28168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" is_pre_election: true
I20260618 13:59:19.078966 28071 leader_election.cc:304] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e4d4f4566be42dfbafb7c0e40fb17ff; no voters: 7637285570574a6dbcef1de6f84ecac0, a950ab7cd66b4cb0b29592aa692c2dd0
I20260618 13:59:19.079399 28735 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:19.084637 28048 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260618 13:59:19.099105 28595 heartbeater.cc:499] Master 127.27.100.60:42083 was elected leader, sending a full tablet report...
I20260618 13:59:19.115705 28229 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_dd36a4c5db7c4e16a1bb76a5bd89f6ad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.117563 28229 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dd36a4c5db7c4e16a1bb76a5bd89f6ad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.118701 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_f40b231e7e2947cab604ac1e583b6e34"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.119320 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f40b231e7e2947cab604ac1e583b6e34 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.119534 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_035f2aea44c340c9b8fd640593e0e9bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.120054 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_035f2aea44c340c9b8fd640593e0e9bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.119933 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_3c272f9a96134d75baa5c0da563e7151"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:19.120538 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_0761fd66624e4bbdb374e6f5c02ebedd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.120816 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3c272f9a96134d75baa5c0da563e7151 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:19.121210 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0761fd66624e4bbdb374e6f5c02ebedd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.121124 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_2e15738790364d8c97db61a7f8f04a0c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:19.120819 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_a8ee858c62224f5b8f7f6491d0daf430"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:19.121585 28225 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_b3f2096e705c477297612dad3199ba91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:19.121820 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_d17be1780e9d4a1eac0b952f6afd6df5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.122179 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e15738790364d8c97db61a7f8f04a0c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.120002 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_4f395883451b46f4ab1514fdfdf094c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.122990 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a8ee858c62224f5b8f7f6491d0daf430 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:19.123196 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d17be1780e9d4a1eac0b952f6afd6df5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:19.125931 28225 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b3f2096e705c477297612dad3199ba91 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:19.128619 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f395883451b46f4ab1514fdfdf094c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.151938 28048 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu pid 28126
I20260618 13:59:19.153671 28188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 13:59:19.160688 28228 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_dd36a4c5db7c4e16a1bb76a5bd89f6ad" } new_table_name: "default.Table_68c3c2222d2446c6a204a0eb163f2cfc" modify_external_catalogs: true
I20260618 13:59:19.161296 28229 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_035f2aea44c340c9b8fd640593e0e9bb" } new_table_name: "default.Table_00e4e33ddb8948a79faa0174798b215f" modify_external_catalogs: true
I20260618 13:59:19.176378 28224 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_0761fd66624e4bbdb374e6f5c02ebedd" } new_table_name: "default.Table_7a5b9ad4d9b44f358878aae89187c35a" modify_external_catalogs: true
I20260618 13:59:19.176635 28220 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_2e15738790364d8c97db61a7f8f04a0c" } new_table_name: "default.Table_8dd5e514382c480eace10306357d5964" modify_external_catalogs: true
I20260618 13:59:19.176580 28223 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_d17be1780e9d4a1eac0b952f6afd6df5" } modify_external_catalogs: true
I20260618 13:59:19.176802 28223 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_d17be1780e9d4a1eac0b952f6afd6df5" } modify_external_catalogs: true
I20260618 13:59:19.177696 28227 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_f40b231e7e2947cab604ac1e583b6e34" } modify_external_catalogs: true
I20260618 13:59:19.177956 28227 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_f40b231e7e2947cab604ac1e583b6e34" } modify_external_catalogs: true
I20260618 13:59:19.177992 28221 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_3c272f9a96134d75baa5c0da563e7151" } modify_external_catalogs: true
I20260618 13:59:19.178162 28221 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_3c272f9a96134d75baa5c0da563e7151" } modify_external_catalogs: true
I20260618 13:59:19.179538 28226 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_a8ee858c62224f5b8f7f6491d0daf430" } new_table_name: "default.Table_15b7dcd63a7747e2b69f95da678dbc61" modify_external_catalogs: true
I20260618 13:59:19.180486 28225 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_b3f2096e705c477297612dad3199ba91" } modify_external_catalogs: true
I20260618 13:59:19.180677 28225 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_b3f2096e705c477297612dad3199ba91" } modify_external_catalogs: true
I20260618 13:59:19.184775 28222 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_4f395883451b46f4ab1514fdfdf094c1" } modify_external_catalogs: true
I20260618 13:59:19.184963 28222 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_4f395883451b46f4ab1514fdfdf094c1" } modify_external_catalogs: true
W20260618 13:59:19.188711 28229 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c1af8b68bd424b92ae7f5a57f8efb5e5 has no consensus state
W20260618 13:59:19.188822 28228 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2efebe57e3d548fa8919221028142bb1 consensus state has no leader
W20260618 13:59:19.223790 28214 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet c1af8b68bd424b92ae7f5a57f8efb5e5 (table default.Table_00e4e33ddb8948a79faa0174798b215f [id=aae6979440544fe984453a915e7c2e00], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet c1af8b68bd424b92ae7f5a57f8efb5e5 consensus state has no leader
I20260618 13:59:19.226099 28221 catalog_manager.cc:5978] Not sending DeleteTablet requests; no consensus state for tablet 0a089547e9874e04b0c8c4a35416d725
I20260618 13:59:19.226298 28223 catalog_manager.cc:5978] Not sending DeleteTablet requests; no consensus state for tablet af73d342063a408baa4db200ac75dd58
I20260618 13:59:19.226644 28227 catalog_manager.cc:5978] Not sending DeleteTablet requests; no consensus state for tablet 5c9aad60db734a3bab5f1e28a043b204
W20260618 13:59:19.226687 28220 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 734374fd92d346c4a9a38c9a3d687df2 has no consensus state
I20260618 13:59:19.234210 28528 tablet_service.cc:1511] Processing CreateTablet for tablet 2efebe57e3d548fa8919221028142bb1 (DEFAULT_TABLE table=default.Table_68c3c2222d2446c6a204a0eb163f2cfc [id=3bcc877eddec4d00b3dc1760cc635b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.235131 28663 tablet_service.cc:1511] Processing CreateTablet for tablet c1af8b68bd424b92ae7f5a57f8efb5e5 (DEFAULT_TABLE table=default.Table_00e4e33ddb8948a79faa0174798b215f [id=aae6979440544fe984453a915e7c2e00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.235759 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2efebe57e3d548fa8919221028142bb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.236586 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1af8b68bd424b92ae7f5a57f8efb5e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.238644 28529 tablet_service.cc:1511] Processing CreateTablet for tablet c1af8b68bd424b92ae7f5a57f8efb5e5 (DEFAULT_TABLE table=default.Table_00e4e33ddb8948a79faa0174798b215f [id=aae6979440544fe984453a915e7c2e00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.238801 28662 tablet_service.cc:1511] Processing CreateTablet for tablet 2efebe57e3d548fa8919221028142bb1 (DEFAULT_TABLE table=default.Table_68c3c2222d2446c6a204a0eb163f2cfc [id=3bcc877eddec4d00b3dc1760cc635b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.239066 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1af8b68bd424b92ae7f5a57f8efb5e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.239269 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2efebe57e3d548fa8919221028142bb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.241293 28225 catalog_manager.cc:5978] Not sending DeleteTablet requests; no consensus state for tablet 1ae3e20ec5a142bb9e3097b2a14a419d
W20260618 13:59:19.241782 28224 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7e82dc44779c451bb7e961e775672de1 has no consensus state
W20260618 13:59:19.244136 28226 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 73f048d041fe4d2e8d944e446cbf099c has no consensus state
I20260618 13:59:19.244525 28222 catalog_manager.cc:5978] Not sending DeleteTablet requests; no consensus state for tablet cce06bb801044a5a8cbd13fad9927338
I20260618 13:59:19.255529 28391 tablet_service.cc:1511] Processing CreateTablet for tablet c1af8b68bd424b92ae7f5a57f8efb5e5 (DEFAULT_TABLE table=default.Table_00e4e33ddb8948a79faa0174798b215f [id=aae6979440544fe984453a915e7c2e00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.257776 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1af8b68bd424b92ae7f5a57f8efb5e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.259079 28390 tablet_service.cc:1511] Processing CreateTablet for tablet 2efebe57e3d548fa8919221028142bb1 (DEFAULT_TABLE table=default.Table_68c3c2222d2446c6a204a0eb163f2cfc [id=3bcc877eddec4d00b3dc1760cc635b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.259754 28390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2efebe57e3d548fa8919221028142bb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.266887 28663 tablet_service.cc:1511] Processing CreateTablet for tablet 73f048d041fe4d2e8d944e446cbf099c (DEFAULT_TABLE table=default.Table_15b7dcd63a7747e2b69f95da678dbc61 [id=474f40a5bbf846d2bd37eeb3acbe8bda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.267441 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f048d041fe4d2e8d944e446cbf099c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.267716 28527 tablet_service.cc:1511] Processing CreateTablet for tablet 7e82dc44779c451bb7e961e775672de1 (DEFAULT_TABLE table=default.Table_7a5b9ad4d9b44f358878aae89187c35a [id=74c1ef2bdbe342ff85c1d825b800a1cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.268298 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e82dc44779c451bb7e961e775672de1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.271462 28661 tablet_service.cc:1511] Processing CreateTablet for tablet 7e82dc44779c451bb7e961e775672de1 (DEFAULT_TABLE table=default.Table_7a5b9ad4d9b44f358878aae89187c35a [id=74c1ef2bdbe342ff85c1d825b800a1cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.272001 28661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e82dc44779c451bb7e961e775672de1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.272205 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 734374fd92d346c4a9a38c9a3d687df2 (DEFAULT_TABLE table=default.Table_8dd5e514382c480eace10306357d5964 [id=a874707bba8d41e7b598d7ad4a3e21f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.272675 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 734374fd92d346c4a9a38c9a3d687df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.283756 28661 tablet_service.cc:1511] Processing CreateTablet for tablet 734374fd92d346c4a9a38c9a3d687df2 (DEFAULT_TABLE table=default.Table_8dd5e514382c480eace10306357d5964 [id=a874707bba8d41e7b598d7ad4a3e21f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.284267 28661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 734374fd92d346c4a9a38c9a3d687df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.295493 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 73f048d041fe4d2e8d944e446cbf099c (DEFAULT_TABLE table=default.Table_15b7dcd63a7747e2b69f95da678dbc61 [id=474f40a5bbf846d2bd37eeb3acbe8bda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.296162 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f048d041fe4d2e8d944e446cbf099c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.301550 28388 tablet_service.cc:1511] Processing CreateTablet for tablet 7e82dc44779c451bb7e961e775672de1 (DEFAULT_TABLE table=default.Table_7a5b9ad4d9b44f358878aae89187c35a [id=74c1ef2bdbe342ff85c1d825b800a1cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.302152 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e82dc44779c451bb7e961e775672de1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.307597 28387 tablet_service.cc:1511] Processing CreateTablet for tablet 734374fd92d346c4a9a38c9a3d687df2 (DEFAULT_TABLE table=default.Table_8dd5e514382c480eace10306357d5964 [id=a874707bba8d41e7b598d7ad4a3e21f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.308189 28387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 734374fd92d346c4a9a38c9a3d687df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.323454 28222 master_service.cc:960] ReplaceTablet: received request to replace tablet 734374fd92d346c4a9a38c9a3d687df2 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:19.328776 28226 master_service.cc:960] ReplaceTablet: received request to replace tablet 7e82dc44779c451bb7e961e775672de1 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:19.329427 28528 tablet_service.cc:1511] Processing CreateTablet for tablet 73f048d041fe4d2e8d944e446cbf099c (DEFAULT_TABLE table=default.Table_15b7dcd63a7747e2b69f95da678dbc61 [id=474f40a5bbf846d2bd37eeb3acbe8bda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.329849 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f048d041fe4d2e8d944e446cbf099c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.329885 28224 master_service.cc:960] ReplaceTablet: received request to replace tablet c1af8b68bd424b92ae7f5a57f8efb5e5 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:19.330518 28225 master_service.cc:960] ReplaceTablet: received request to replace tablet 7e82dc44779c451bb7e961e775672de1 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:19.333562 28223 master_service.cc:960] ReplaceTablet: received request to replace tablet 7e82dc44779c451bb7e961e775672de1 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:19.337584 28798 tablet_bootstrap.cc:492] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:19.337867 28222 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 734374fd92d346c4a9a38c9a3d687df2
I20260618 13:59:19.338394 28226 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 7e82dc44779c451bb7e961e775672de1
I20260618 13:59:19.338658 28222 catalog_manager.cc:6784] ReplaceTablet: tablet 734374fd92d346c4a9a38c9a3d687df2 deleted and replaced by tablet 68b3bb6a4dd3446288822b5424fe5122
I20260618 13:59:19.338716 28224 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet c1af8b68bd424b92ae7f5a57f8efb5e5
I20260618 13:59:19.338932 28226 catalog_manager.cc:6784] ReplaceTablet: tablet 7e82dc44779c451bb7e961e775672de1 deleted and replaced by tablet 657635f877e24b76879843d61b55dd62
I20260618 13:59:19.339169 28224 catalog_manager.cc:6784] ReplaceTablet: tablet c1af8b68bd424b92ae7f5a57f8efb5e5 deleted and replaced by tablet fc685eec02804dfabe2e25405b96def3
I20260618 13:59:19.340381 28525 tablet_service.cc:1558] Processing DeleteTablet for tablet c1af8b68bd424b92ae7f5a57f8efb5e5 with delete_type TABLET_DATA_DELETED (replaced by tablet fc685eec02804dfabe2e25405b96def3) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:19.340386 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet c1af8b68bd424b92ae7f5a57f8efb5e5 with delete_type TABLET_DATA_DELETED (replaced by tablet fc685eec02804dfabe2e25405b96def3) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.340592 28663 tablet_service.cc:1558] Processing DeleteTablet for tablet 734374fd92d346c4a9a38c9a3d687df2 with delete_type TABLET_DATA_DELETED (replaced by tablet 68b3bb6a4dd3446288822b5424fe5122) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.340452 28661 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e82dc44779c451bb7e961e775672de1 with delete_type TABLET_DATA_DELETED (replaced by tablet 657635f877e24b76879843d61b55dd62) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.341131 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet 734374fd92d346c4a9a38c9a3d687df2 with delete_type TABLET_DATA_DELETED (replaced by tablet 68b3bb6a4dd3446288822b5424fe5122) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:19.341737 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e82dc44779c451bb7e961e775672de1 with delete_type TABLET_DATA_DELETED (replaced by tablet 657635f877e24b76879843d61b55dd62) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:19.343070 28388 tablet_service.cc:1558] Processing DeleteTablet for tablet c1af8b68bd424b92ae7f5a57f8efb5e5 with delete_type TABLET_DATA_DELETED (replaced by tablet fc685eec02804dfabe2e25405b96def3) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:19.346504 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet c1af8b68bd424b92ae7f5a57f8efb5e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c1af8b68bd424b92ae7f5a57f8efb5e5 already in progress: creating tablet
I20260618 13:59:19.346715 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e82dc44779c451bb7e961e775672de1 with delete_type TABLET_DATA_DELETED (replaced by tablet 657635f877e24b76879843d61b55dd62) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:19.347323 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet 734374fd92d346c4a9a38c9a3d687df2 with delete_type TABLET_DATA_DELETED (replaced by tablet 68b3bb6a4dd3446288822b5424fe5122) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:19.347931 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 7e82dc44779c451bb7e961e775672de1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e82dc44779c451bb7e961e775672de1 already in progress: creating tablet
W20260618 13:59:19.348148 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 734374fd92d346c4a9a38c9a3d687df2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 734374fd92d346c4a9a38c9a3d687df2 already in progress: creating tablet
W20260618 13:59:19.349237 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet c1af8b68bd424b92ae7f5a57f8efb5e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c1af8b68bd424b92ae7f5a57f8efb5e5 already in progress: creating tablet
W20260618 13:59:19.349483 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 734374fd92d346c4a9a38c9a3d687df2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 734374fd92d346c4a9a38c9a3d687df2 already in progress: creating tablet
W20260618 13:59:19.349632 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 7e82dc44779c451bb7e961e775672de1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e82dc44779c451bb7e961e775672de1 already in progress: creating tablet
W20260618 13:59:19.350294 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 734374fd92d346c4a9a38c9a3d687df2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 734374fd92d346c4a9a38c9a3d687df2 already in progress: creating tablet
W20260618 13:59:19.350739 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 7e82dc44779c451bb7e961e775672de1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e82dc44779c451bb7e961e775672de1 already in progress: creating tablet
W20260618 13:59:19.350991 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet c1af8b68bd424b92ae7f5a57f8efb5e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c1af8b68bd424b92ae7f5a57f8efb5e5 already in progress: creating tablet
I20260618 13:59:19.351804 28799 tablet_bootstrap.cc:492] T 73f048d041fe4d2e8d944e446cbf099c P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:19.355566 28798 tablet_bootstrap.cc:654] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.355581 28799 tablet_bootstrap.cc:654] T 73f048d041fe4d2e8d944e446cbf099c P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.356340 28797 tablet_bootstrap.cc:492] T 2efebe57e3d548fa8919221028142bb1 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:19.356926 28799 log.cc:826] T 73f048d041fe4d2e8d944e446cbf099c P 6fc808af0e884ed8836561665d8e0e7b: Log is configured to *not* fsync() on all Append() calls
I20260618 13:59:19.357558 28663 tablet_service.cc:1511] Processing CreateTablet for tablet 657635f877e24b76879843d61b55dd62 (DEFAULT_TABLE table=default.Table_7a5b9ad4d9b44f358878aae89187c35a [id=74c1ef2bdbe342ff85c1d825b800a1cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.357978 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 657635f877e24b76879843d61b55dd62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.358227 28525 tablet_service.cc:1511] Processing CreateTablet for tablet fc685eec02804dfabe2e25405b96def3 (DEFAULT_TABLE table=default.Table_00e4e33ddb8948a79faa0174798b215f [id=aae6979440544fe984453a915e7c2e00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.358548 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_9b9cd88f00b34896aab534454c799e62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:19.358747 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc685eec02804dfabe2e25405b96def3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:19.359143 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9b9cd88f00b34896aab534454c799e62 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.359843 28798 log.cc:826] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 97859748af2c41879ba2a668dda999d2: Log is configured to *not* fsync() on all Append() calls
I20260618 13:59:19.360457 28225 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_e7787f99590e428ca58ae371cdfeffd5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.360983 28225 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e7787f99590e428ca58ae371cdfeffd5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.362682 28797 tablet_bootstrap.cc:654] T 2efebe57e3d548fa8919221028142bb1 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.363807 28528 tablet_service.cc:1511] Processing CreateTablet for tablet 657635f877e24b76879843d61b55dd62 (DEFAULT_TABLE table=default.Table_7a5b9ad4d9b44f358878aae89187c35a [id=74c1ef2bdbe342ff85c1d825b800a1cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.364177 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 657635f877e24b76879843d61b55dd62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.364331 28797 log.cc:826] T 2efebe57e3d548fa8919221028142bb1 P d6a169b16ced46ca9cfbc631709bb257: Log is configured to *not* fsync() on all Append() calls
I20260618 13:59:19.366096 28660 tablet_service.cc:1511] Processing CreateTablet for tablet fc685eec02804dfabe2e25405b96def3 (DEFAULT_TABLE table=default.Table_00e4e33ddb8948a79faa0174798b215f [id=aae6979440544fe984453a915e7c2e00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.366572 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc685eec02804dfabe2e25405b96def3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.371484 28225 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_75a78191557544468211aa80f08fb3a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.371999 28225 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_75a78191557544468211aa80f08fb3a2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.373306 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_08cb9bd9bbb442648c47edbdd3a13ecd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.373754 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_08cb9bd9bbb442648c47edbdd3a13ecd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.374162 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_e3fc60c072be498795f1f0e1baa81476"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:19.374279 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_b964eadba5974835bbacd9c0b049aaf3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.374707 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b964eadba5974835bbacd9c0b049aaf3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:19.374765 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e3fc60c072be498795f1f0e1baa81476 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.375244 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 657635f877e24b76879843d61b55dd62 (DEFAULT_TABLE table=default.Table_7a5b9ad4d9b44f358878aae89187c35a [id=74c1ef2bdbe342ff85c1d825b800a1cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.375653 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 657635f877e24b76879843d61b55dd62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.382467 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_0700cfccef1048aea83be790024db03d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.382965 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0700cfccef1048aea83be790024db03d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.384212 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_1cdc3c58c3964fd6a6f8137d22460d61"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.384773 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1cdc3c58c3964fd6a6f8137d22460d61 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.384714 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_3b5a67572c914054908f9bde6bb0ee09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.385217 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3b5a67572c914054908f9bde6bb0ee09 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.385223 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_7559b439752d4fa68c4fb6afee136bfc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.387187 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7559b439752d4fa68c4fb6afee136bfc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.395964 28226 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_75a78191557544468211aa80f08fb3a2" } new_table_name: "default.Table_e10ecf3a190148d0b17f97204f155353" modify_external_catalogs: true
I20260618 13:59:19.398342 28797 tablet_bootstrap.cc:492] T 2efebe57e3d548fa8919221028142bb1 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:19.398736 28797 ts_tablet_manager.cc:1403] T 2efebe57e3d548fa8919221028142bb1 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.043s user 0.005s sys 0.016s
I20260618 13:59:19.396014 28223 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_e3fc60c072be498795f1f0e1baa81476" } new_table_name: "default.Table_64a3f247686c4936bcf839716c046ff6" modify_external_catalogs: true
W20260618 13:59:19.409763 28226 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 has no consensus state
W20260618 13:59:19.410257 28223 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 397c7abe26f4451298558d6a0875b677 has no consensus state
I20260618 13:59:19.411927 28797 raft_consensus.cc:359] T 2efebe57e3d548fa8919221028142bb1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.412614 28797 raft_consensus.cc:385] T 2efebe57e3d548fa8919221028142bb1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.412775 28797 raft_consensus.cc:740] T 2efebe57e3d548fa8919221028142bb1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:19.413687 28797 consensus_queue.cc:260] T 2efebe57e3d548fa8919221028142bb1 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.413844 28389 tablet_service.cc:1511] Processing CreateTablet for tablet fc685eec02804dfabe2e25405b96def3 (DEFAULT_TABLE table=default.Table_00e4e33ddb8948a79faa0174798b215f [id=aae6979440544fe984453a915e7c2e00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.414402 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc685eec02804dfabe2e25405b96def3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.417393 28799 tablet_bootstrap.cc:492] T 73f048d041fe4d2e8d944e446cbf099c P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:19.417800 28799 ts_tablet_manager.cc:1403] T 73f048d041fe4d2e8d944e446cbf099c P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.066s user 0.006s sys 0.006s
I20260618 13:59:19.420713 28529 tablet_service.cc:1511] Processing CreateTablet for tablet 68b3bb6a4dd3446288822b5424fe5122 (DEFAULT_TABLE table=default.Table_8dd5e514382c480eace10306357d5964 [id=a874707bba8d41e7b598d7ad4a3e21f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.421121 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68b3bb6a4dd3446288822b5424fe5122. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.421109 28662 tablet_service.cc:1511] Processing CreateTablet for tablet 68b3bb6a4dd3446288822b5424fe5122 (DEFAULT_TABLE table=default.Table_8dd5e514382c480eace10306357d5964 [id=a874707bba8d41e7b598d7ad4a3e21f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.421466 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68b3bb6a4dd3446288822b5424fe5122. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.422745 28391 tablet_service.cc:1511] Processing CreateTablet for tablet 68b3bb6a4dd3446288822b5424fe5122 (DEFAULT_TABLE table=default.Table_8dd5e514382c480eace10306357d5964 [id=a874707bba8d41e7b598d7ad4a3e21f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.423159 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68b3bb6a4dd3446288822b5424fe5122. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.428793 28223 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_e10ecf3a190148d0b17f97204f155353" } modify_external_catalogs: true
I20260618 13:59:19.428917 28226 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_7559b439752d4fa68c4fb6afee136bfc" } modify_external_catalogs: true
I20260618 13:59:19.428995 28223 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_e10ecf3a190148d0b17f97204f155353" } modify_external_catalogs: true
I20260618 13:59:19.429148 28226 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_7559b439752d4fa68c4fb6afee136bfc" } modify_external_catalogs: true
I20260618 13:59:19.429963 28222 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_68c3c2222d2446c6a204a0eb163f2cfc" } new_table_name: "default.Table_fced244393ee4bd3ac9796af6863c9dc" modify_external_catalogs: true
I20260618 13:59:19.430053 28221 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_0700cfccef1048aea83be790024db03d" } modify_external_catalogs: true
I20260618 13:59:19.430205 28221 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_0700cfccef1048aea83be790024db03d" } modify_external_catalogs: true
I20260618 13:59:19.430979 28799 raft_consensus.cc:359] T 73f048d041fe4d2e8d944e446cbf099c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.431382 28799 raft_consensus.cc:385] T 73f048d041fe4d2e8d944e446cbf099c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.431495 28799 raft_consensus.cc:740] T 73f048d041fe4d2e8d944e446cbf099c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
W20260618 13:59:19.436501 28222 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2efebe57e3d548fa8919221028142bb1 consensus state has no leader
I20260618 13:59:19.436764 28797 ts_tablet_manager.cc:1434] T 2efebe57e3d548fa8919221028142bb1 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.038s user 0.005s sys 0.014s
I20260618 13:59:19.437089 28729 heartbeater.cc:499] Master 127.27.100.60:42083 was elected leader, sending a full tablet report...
I20260618 13:59:19.438495 28799 consensus_queue.cc:260] T 73f048d041fe4d2e8d944e446cbf099c P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.440769 28797 tablet_bootstrap.cc:492] T c1af8b68bd424b92ae7f5a57f8efb5e5 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:19.442524 28797 tablet_bootstrap.cc:654] T c1af8b68bd424b92ae7f5a57f8efb5e5 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.443857 28227 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_15b7dcd63a7747e2b69f95da678dbc61" } modify_external_catalogs: true
I20260618 13:59:19.444103 28227 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_15b7dcd63a7747e2b69f95da678dbc61" } modify_external_catalogs: true
I20260618 13:59:19.444830 28220 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_64a3f247686c4936bcf839716c046ff6" } new_table_name: "default.Table_a3eb0e4580814fc392f14934ec66ed3c" modify_external_catalogs: true
I20260618 13:59:19.446481 28225 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_9b9cd88f00b34896aab534454c799e62" } new_table_name: "default.Table_1c9a4bfc47d4424aab1b47c294cc1ff5" modify_external_catalogs: true
I20260618 13:59:19.450157 28224 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_7a5b9ad4d9b44f358878aae89187c35a" } modify_external_catalogs: true
I20260618 13:59:19.450320 28224 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_7a5b9ad4d9b44f358878aae89187c35a" } modify_external_catalogs: true
I20260618 13:59:19.451576 28798 tablet_bootstrap.cc:492] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:19.451970 28798 ts_tablet_manager.cc:1403] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.115s user 0.028s sys 0.026s
W20260618 13:59:19.453356 28220 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 397c7abe26f4451298558d6a0875b677 has no consensus state
I20260618 13:59:19.454896 28799 ts_tablet_manager.cc:1434] T 73f048d041fe4d2e8d944e446cbf099c P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.037s user 0.015s sys 0.002s
I20260618 13:59:19.458827 28799 tablet_bootstrap.cc:492] T 7e82dc44779c451bb7e961e775672de1 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:19.461063 28799 tablet_bootstrap.cc:654] T 7e82dc44779c451bb7e961e775672de1 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.461861 28798 raft_consensus.cc:359] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:19.462458 28798 raft_consensus.cc:385] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.462620 28798 raft_consensus.cc:740] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:19.463526 28798 consensus_queue.cc:260] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:19.463773 28225 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 59613383d2294f9ab6bce6321aad44bb has no consensus state
I20260618 13:59:19.473992 28389 tablet_service.cc:1511] Processing CreateTablet for tablet a6189fed26fb471c9724fd594054a58c (DEFAULT_TABLE table=default.Table_1cdc3c58c3964fd6a6f8137d22460d61 [id=979317e20c204b109fb05c4128c0e2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.474200 28391 tablet_service.cc:1511] Processing CreateTablet for tablet fcc9e4252a1d453192e429ce4d58dfe1 (DEFAULT_TABLE table=default.Table_0700cfccef1048aea83be790024db03d [id=018abd3973f146e9880ac9ba37ddfd20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.474475 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6189fed26fb471c9724fd594054a58c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.474788 28390 tablet_service.cc:1511] Processing CreateTablet for tablet 59613383d2294f9ab6bce6321aad44bb (DEFAULT_TABLE table=default.Table_1c9a4bfc47d4424aab1b47c294cc1ff5 [id=7923680bde144e3dbb78c922d09540b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.475104 28388 tablet_service.cc:1511] Processing CreateTablet for tablet 07a4e95bd6ae47beac34fd36d6e956e3 (DEFAULT_TABLE table=default.Table_b964eadba5974835bbacd9c0b049aaf3 [id=972bc2c124ac459da06ec4093c5b564b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.475569 28390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59613383d2294f9ab6bce6321aad44bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.476261 28221 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet fcc9e4252a1d453192e429ce4d58dfe1
I20260618 13:59:19.477216 28223 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 9fd3fbcf0d9745a1b3d3408f95022bf6
I20260618 13:59:19.478283 28226 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet f7027a4138784d029255fc89c73104a3
I20260618 13:59:19.478569 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07a4e95bd6ae47beac34fd36d6e956e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.480597 28528 tablet_service.cc:1511] Processing CreateTablet for tablet a6189fed26fb471c9724fd594054a58c (DEFAULT_TABLE table=default.Table_1cdc3c58c3964fd6a6f8137d22460d61 [id=979317e20c204b109fb05c4128c0e2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.480785 28662 tablet_service.cc:1511] Processing CreateTablet for tablet a6189fed26fb471c9724fd594054a58c (DEFAULT_TABLE table=default.Table_1cdc3c58c3964fd6a6f8137d22460d61 [id=979317e20c204b109fb05c4128c0e2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.481125 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcc9e4252a1d453192e429ce4d58dfe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.481189 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6189fed26fb471c9724fd594054a58c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.481595 28525 tablet_service.cc:1511] Processing CreateTablet for tablet fcc9e4252a1d453192e429ce4d58dfe1 (DEFAULT_TABLE table=default.Table_0700cfccef1048aea83be790024db03d [id=018abd3973f146e9880ac9ba37ddfd20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.481902 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6189fed26fb471c9724fd594054a58c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.482542 28386 tablet_service.cc:1511] Processing CreateTablet for tablet f7027a4138784d029255fc89c73104a3 (DEFAULT_TABLE table=default.Table_7559b439752d4fa68c4fb6afee136bfc [id=61b2edf42a374060934bf910c3f2dde5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.483213 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7027a4138784d029255fc89c73104a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.486644 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcc9e4252a1d453192e429ce4d58dfe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.486644 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet f7027a4138784d029255fc89c73104a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:19.486716 28663 tablet_service.cc:1511] Processing CreateTablet for tablet fcc9e4252a1d453192e429ce4d58dfe1 (DEFAULT_TABLE table=default.Table_0700cfccef1048aea83be790024db03d [id=018abd3973f146e9880ac9ba37ddfd20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.487107 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcc9e4252a1d453192e429ce4d58dfe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.487564 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:19.488125 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 397c7abe26f4451298558d6a0875b677 (DEFAULT_TABLE table=default.Table_a3eb0e4580814fc392f14934ec66ed3c [id=ce780967c12a412a8d7f0bef2f9c00e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.488682 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 397c7abe26f4451298558d6a0875b677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.482548 28387 tablet_service.cc:1558] Processing DeleteTablet for tablet fcc9e4252a1d453192e429ce4d58dfe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:19.491744 28214 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet f7027a4138784d029255fc89c73104a3 because the tablet was not found. No further retry: Not found: Tablet not found: f7027a4138784d029255fc89c73104a3
W20260618 13:59:19.492015 28214 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 because the tablet was not found. No further retry: Not found: Tablet not found: 9fd3fbcf0d9745a1b3d3408f95022bf6
W20260618 13:59:19.492250 28214 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet fcc9e4252a1d453192e429ce4d58dfe1 because the tablet was not found. No further retry: Not found: Tablet not found: fcc9e4252a1d453192e429ce4d58dfe1
I20260618 13:59:19.494439 28387 tablet_service.cc:1511] Processing CreateTablet for tablet f4447a3e87bd476e84f59dd4c4185b26 (DEFAULT_TABLE table=default.Table_08cb9bd9bbb442648c47edbdd3a13ecd [id=80246ec4fbca44248528c2c379034cee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.494916 28387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4447a3e87bd476e84f59dd4c4185b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.500300 28528 tablet_service.cc:1511] Processing CreateTablet for tablet 59613383d2294f9ab6bce6321aad44bb (DEFAULT_TABLE table=default.Table_1c9a4bfc47d4424aab1b47c294cc1ff5 [id=7923680bde144e3dbb78c922d09540b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.500664 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59613383d2294f9ab6bce6321aad44bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.501205 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet fcc9e4252a1d453192e429ce4d58dfe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.501928 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.502836 28661 tablet_service.cc:1558] Processing DeleteTablet for tablet f7027a4138784d029255fc89c73104a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.503383 28660 tablet_service.cc:1511] Processing CreateTablet for tablet f7027a4138784d029255fc89c73104a3 (DEFAULT_TABLE table=default.Table_7559b439752d4fa68c4fb6afee136bfc [id=61b2edf42a374060934bf910c3f2dde5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.503398 28525 tablet_service.cc:1511] Processing CreateTablet for tablet 07a4e95bd6ae47beac34fd36d6e956e3 (DEFAULT_TABLE table=default.Table_b964eadba5974835bbacd9c0b049aaf3 [id=972bc2c124ac459da06ec4093c5b564b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.503747 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7027a4138784d029255fc89c73104a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.503747 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07a4e95bd6ae47beac34fd36d6e956e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.506538 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet fcc9e4252a1d453192e429ce4d58dfe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:19.506824 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:19.507184 28526 tablet_service.cc:1511] Processing CreateTablet for tablet f7027a4138784d029255fc89c73104a3 (DEFAULT_TABLE table=default.Table_7559b439752d4fa68c4fb6afee136bfc [id=61b2edf42a374060934bf910c3f2dde5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.507510 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7027a4138784d029255fc89c73104a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.507800 28799 tablet_bootstrap.cc:492] T 7e82dc44779c451bb7e961e775672de1 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:19.507966 28799 ts_tablet_manager.cc:1403] T 7e82dc44779c451bb7e961e775672de1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.049s user 0.005s sys 0.001s
I20260618 13:59:19.509155 28799 raft_consensus.cc:359] T 7e82dc44779c451bb7e961e775672de1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.509366 28799 raft_consensus.cc:385] T 7e82dc44779c451bb7e961e775672de1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.509472 28799 raft_consensus.cc:740] T 7e82dc44779c451bb7e961e775672de1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:19.509815 28799 consensus_queue.cc:260] T 7e82dc44779c451bb7e961e775672de1 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.510106 28662 tablet_service.cc:1511] Processing CreateTablet for tablet 59613383d2294f9ab6bce6321aad44bb (DEFAULT_TABLE table=default.Table_1c9a4bfc47d4424aab1b47c294cc1ff5 [id=7923680bde144e3dbb78c922d09540b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.510594 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet f7027a4138784d029255fc89c73104a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:19.510995 28527 tablet_service.cc:1511] Processing CreateTablet for tablet 397c7abe26f4451298558d6a0875b677 (DEFAULT_TABLE table=default.Table_a3eb0e4580814fc392f14934ec66ed3c [id=ce780967c12a412a8d7f0bef2f9c00e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.511317 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 397c7abe26f4451298558d6a0875b677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.511507 28799 ts_tablet_manager.cc:1434] T 7e82dc44779c451bb7e961e775672de1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260618 13:59:19.511837 28799 tablet_bootstrap.cc:492] T 734374fd92d346c4a9a38c9a3d687df2 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:19.510131 28661 tablet_service.cc:1511] Processing CreateTablet for tablet 397c7abe26f4451298558d6a0875b677 (DEFAULT_TABLE table=default.Table_a3eb0e4580814fc392f14934ec66ed3c [id=ce780967c12a412a8d7f0bef2f9c00e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.512234 28661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 397c7abe26f4451298558d6a0875b677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.513809 28799 tablet_bootstrap.cc:654] T 734374fd92d346c4a9a38c9a3d687df2 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:19.515980 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet fcc9e4252a1d453192e429ce4d58dfe1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fcc9e4252a1d453192e429ce4d58dfe1 already in progress: creating tablet
W20260618 13:59:19.517261 28215 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 because the tablet was not found. No further retry: Not found: Tablet not found: 9fd3fbcf0d9745a1b3d3408f95022bf6
W20260618 13:59:19.517639 28215 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet f7027a4138784d029255fc89c73104a3 because the tablet was not found. No further retry: Not found: Tablet not found: f7027a4138784d029255fc89c73104a3
I20260618 13:59:19.509810 28663 tablet_service.cc:1511] Processing CreateTablet for tablet 07a4e95bd6ae47beac34fd36d6e956e3 (DEFAULT_TABLE table=default.Table_b964eadba5974835bbacd9c0b049aaf3 [id=972bc2c124ac459da06ec4093c5b564b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:19.518252 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet fcc9e4252a1d453192e429ce4d58dfe1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fcc9e4252a1d453192e429ce4d58dfe1 already in progress: creating tablet
I20260618 13:59:19.518366 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07a4e95bd6ae47beac34fd36d6e956e3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:19.518608 28215 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 because the tablet was not found. No further retry: Not found: Tablet not found: 9fd3fbcf0d9745a1b3d3408f95022bf6
W20260618 13:59:19.518878 28215 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet f7027a4138784d029255fc89c73104a3 because the tablet was not found. No further retry: Not found: Tablet not found: f7027a4138784d029255fc89c73104a3
I20260618 13:59:19.521294 28222 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet fc685eec02804dfabe2e25405b96def3 (table default.Table_00e4e33ddb8948a79faa0174798b215f [id=aae6979440544fe984453a915e7c2e00]). Expected version 1 got 0
I20260618 13:59:19.521610 28222 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 2efebe57e3d548fa8919221028142bb1 (table default.Table_fced244393ee4bd3ac9796af6863c9dc [id=3bcc877eddec4d00b3dc1760cc635b36]). Expected version 2 got 0
I20260618 13:59:19.521786 28222 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 68b3bb6a4dd3446288822b5424fe5122 (table default.Table_8dd5e514382c480eace10306357d5964 [id=a874707bba8d41e7b598d7ad4a3e21f3]). Expected version 1 got 0
I20260618 13:59:19.522701 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59613383d2294f9ab6bce6321aad44bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.523100 28224 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 657635f877e24b76879843d61b55dd62
I20260618 13:59:19.524596 28227 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 73f048d041fe4d2e8d944e446cbf099c
W20260618 13:59:19.527310 28222 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet fc685eec02804dfabe2e25405b96def3 (table default.Table_00e4e33ddb8948a79faa0174798b215f [id=aae6979440544fe984453a915e7c2e00], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet fc685eec02804dfabe2e25405b96def3 consensus state has no leader
W20260618 13:59:19.530267 28222 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet 68b3bb6a4dd3446288822b5424fe5122 (table default.Table_8dd5e514382c480eace10306357d5964 [id=a874707bba8d41e7b598d7ad4a3e21f3], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 68b3bb6a4dd3446288822b5424fe5122 consensus state has no leader
I20260618 13:59:19.530844 28048 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu with pid 28126
I20260618 13:59:19.531515 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 73f048d041fe4d2e8d944e446cbf099c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:19.531776 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet 657635f877e24b76879843d61b55dd62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:19.532114 28802 tablet_replica.cc:333] T 73f048d041fe4d2e8d944e446cbf099c P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:19.532667 28802 raft_consensus.cc:2243] T 73f048d041fe4d2e8d944e446cbf099c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:19.533030 28802 raft_consensus.cc:2272] T 73f048d041fe4d2e8d944e446cbf099c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:19.534440 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet 657635f877e24b76879843d61b55dd62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:19.535239 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet 73f048d041fe4d2e8d944e446cbf099c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:19.535794 28659 tablet_service.cc:1558] Processing DeleteTablet for tablet c1af8b68bd424b92ae7f5a57f8efb5e5 with delete_type TABLET_DATA_DELETED (replaced by tablet fc685eec02804dfabe2e25405b96def3) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.536233 28658 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e82dc44779c451bb7e961e775672de1 with delete_type TABLET_DATA_DELETED (replaced by tablet 657635f877e24b76879843d61b55dd62) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.537776 28658 tablet_service.cc:1558] Processing DeleteTablet for tablet 734374fd92d346c4a9a38c9a3d687df2 with delete_type TABLET_DATA_DELETED (replaced by tablet 68b3bb6a4dd3446288822b5424fe5122) from {username='slave'} at 127.0.0.1:43976
W20260618 13:59:19.538205 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 657635f877e24b76879843d61b55dd62 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 657635f877e24b76879843d61b55dd62 already in progress: creating tablet
W20260618 13:59:19.538583 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 73f048d041fe4d2e8d944e446cbf099c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 73f048d041fe4d2e8d944e446cbf099c already in progress: creating tablet
W20260618 13:59:19.539451 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet c1af8b68bd424b92ae7f5a57f8efb5e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c1af8b68bd424b92ae7f5a57f8efb5e5 already in progress: creating tablet
W20260618 13:59:19.539820 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 7e82dc44779c451bb7e961e775672de1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e82dc44779c451bb7e961e775672de1 already in progress: creating tablet
W20260618 13:59:19.540132 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 734374fd92d346c4a9a38c9a3d687df2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 734374fd92d346c4a9a38c9a3d687df2 already in progress: creating tablet
I20260618 13:59:19.540630 28661 tablet_service.cc:1558] Processing DeleteTablet for tablet 657635f877e24b76879843d61b55dd62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.541267 28802 ts_tablet_manager.cc:1916] T 73f048d041fe4d2e8d944e446cbf099c P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:19.541342 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet 73f048d041fe4d2e8d944e446cbf099c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
W20260618 13:59:19.541316 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 657635f877e24b76879843d61b55dd62 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 657635f877e24b76879843d61b55dd62 already in progress: creating tablet
W20260618 13:59:19.541955 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 73f048d041fe4d2e8d944e446cbf099c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 73f048d041fe4d2e8d944e446cbf099c already in progress: creating tablet
I20260618 13:59:19.542003 28387 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e82dc44779c451bb7e961e775672de1 with delete_type TABLET_DATA_DELETED (replaced by tablet 657635f877e24b76879843d61b55dd62) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:19.551849 28220 master_service.cc:960] ReplaceTablet: received request to replace tablet 2efebe57e3d548fa8919221028142bb1 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:19.552955 28222 master_service.cc:960] ReplaceTablet: received request to replace tablet f7027a4138784d029255fc89c73104a3 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:19.554605 28797 tablet_bootstrap.cc:492] T c1af8b68bd424b92ae7f5a57f8efb5e5 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:19.554754 28797 ts_tablet_manager.cc:1403] T c1af8b68bd424b92ae7f5a57f8efb5e5 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.114s user 0.004s sys 0.017s
I20260618 13:59:19.556124 28802 ts_tablet_manager.cc:1929] T 73f048d041fe4d2e8d944e446cbf099c P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:19.556298 28802 log.cc:1199] T 73f048d041fe4d2e8d944e446cbf099c P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/73f048d041fe4d2e8d944e446cbf099c
I20260618 13:59:19.556795 28802 ts_tablet_manager.cc:1950] T 73f048d041fe4d2e8d944e446cbf099c P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
W20260618 13:59:19.558053 28731 tablet.cc:2406] T 2efebe57e3d548fa8919221028142bb1 P d6a169b16ced46ca9cfbc631709bb257: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 13:59:19.558807 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 73f048d041fe4d2e8d944e446cbf099c (table default.Table_15b7dcd63a7747e2b69f95da678dbc61 [id=474f40a5bbf846d2bd37eeb3acbe8bda]) successfully deleted
I20260618 13:59:19.560251 28799 tablet_bootstrap.cc:492] T 734374fd92d346c4a9a38c9a3d687df2 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:19.560319 28222 master_service.cc:960] ReplaceTablet: received request to replace tablet 7e82dc44779c451bb7e961e775672de1 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:19.560443 28799 ts_tablet_manager.cc:1403] T 734374fd92d346c4a9a38c9a3d687df2 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.049s user 0.001s sys 0.006s
I20260618 13:59:19.560702 28227 master_service.cc:960] ReplaceTablet: received request to replace tablet 7e82dc44779c451bb7e961e775672de1 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:19.560809 28797 raft_consensus.cc:359] T c1af8b68bd424b92ae7f5a57f8efb5e5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:19.560971 28797 raft_consensus.cc:385] T c1af8b68bd424b92ae7f5a57f8efb5e5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.561038 28797 raft_consensus.cc:740] T c1af8b68bd424b92ae7f5a57f8efb5e5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:19.561286 28799 raft_consensus.cc:359] T 734374fd92d346c4a9a38c9a3d687df2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.561506 28799 raft_consensus.cc:385] T 734374fd92d346c4a9a38c9a3d687df2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.561666 28227 master_service.cc:960] ReplaceTablet: received request to replace tablet 73f048d041fe4d2e8d944e446cbf099c from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:19.561609 28799 raft_consensus.cc:740] T 734374fd92d346c4a9a38c9a3d687df2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:19.562376 28799 consensus_queue.cc:260] T 734374fd92d346c4a9a38c9a3d687df2 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.562907 28387 tablet_service.cc:1511] Processing CreateTablet for tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 (DEFAULT_TABLE table=default.Table_e10ecf3a190148d0b17f97204f155353 [id=179292b8d0574f7bb5b6edb5dcad5cb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.563166 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 91c250dfbd72416d9d74f6cbb67b2bdf (DEFAULT_TABLE table=default.Table_3b5a67572c914054908f9bde6bb0ee09 [id=95f51150591e45e8a69bfacc9ac17f24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.563220 28797 consensus_queue.cc:260] T c1af8b68bd424b92ae7f5a57f8efb5e5 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:19.563709 28797 ts_tablet_manager.cc:1434] T c1af8b68bd424b92ae7f5a57f8efb5e5 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260618 13:59:19.566370 28799 ts_tablet_manager.cc:1434] T 734374fd92d346c4a9a38c9a3d687df2 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260618 13:59:19.567305 28797 tablet_bootstrap.cc:492] T 7e82dc44779c451bb7e961e775672de1 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:19.567425 28527 tablet_service.cc:1511] Processing CreateTablet for tablet f4447a3e87bd476e84f59dd4c4185b26 (DEFAULT_TABLE table=default.Table_08cb9bd9bbb442648c47edbdd3a13ecd [id=80246ec4fbca44248528c2c379034cee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.567809 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4447a3e87bd476e84f59dd4c4185b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.569757 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet 734374fd92d346c4a9a38c9a3d687df2 with delete_type TABLET_DATA_DELETED (replaced by tablet 68b3bb6a4dd3446288822b5424fe5122) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:19.572822 28797 tablet_bootstrap.cc:654] T 7e82dc44779c451bb7e961e775672de1 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.574564 28386 tablet_service.cc:1511] Processing CreateTablet for tablet 587b82a6da8d4fe48b26070803831d9d (DEFAULT_TABLE table=default.Table_e7787f99590e428ca58ae371cdfeffd5 [id=262236d66ee24ad8a9e4e27b4b1c3ac1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.575897 28798 ts_tablet_manager.cc:1434] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.124s user 0.027s sys 0.033s
I20260618 13:59:19.576188 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91c250dfbd72416d9d74f6cbb67b2bdf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:19.576308 28597 tablet.cc:2406] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 97859748af2c41879ba2a668dda999d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 13:59:19.578827 28387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fd3fbcf0d9745a1b3d3408f95022bf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.578950 28798 tablet_bootstrap.cc:492] T 2efebe57e3d548fa8919221028142bb1 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:19.579487 28799 tablet_bootstrap.cc:492] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:19.581418 28799 tablet_bootstrap.cc:654] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.582633 28798 tablet_bootstrap.cc:654] T 2efebe57e3d548fa8919221028142bb1 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.582646 28802 tablet_replica.cc:333] T 7e82dc44779c451bb7e961e775672de1 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
W20260618 13:59:19.582808 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 657635f877e24b76879843d61b55dd62 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 657635f877e24b76879843d61b55dd62 already in progress: creating tablet
I20260618 13:59:19.582854 28802 raft_consensus.cc:2243] T 7e82dc44779c451bb7e961e775672de1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:19.583027 28802 raft_consensus.cc:2272] T 7e82dc44779c451bb7e961e775672de1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:19.587999 28802 ts_tablet_manager.cc:1916] T 7e82dc44779c451bb7e961e775672de1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:19.591607 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 587b82a6da8d4fe48b26070803831d9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.599740 28662 tablet_service.cc:1511] Processing CreateTablet for tablet f4447a3e87bd476e84f59dd4c4185b26 (DEFAULT_TABLE table=default.Table_08cb9bd9bbb442648c47edbdd3a13ecd [id=80246ec4fbca44248528c2c379034cee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.600060 28527 tablet_service.cc:1511] Processing CreateTablet for tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 (DEFAULT_TABLE table=default.Table_e10ecf3a190148d0b17f97204f155353 [id=179292b8d0574f7bb5b6edb5dcad5cb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.600189 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4447a3e87bd476e84f59dd4c4185b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.600435 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fd3fbcf0d9745a1b3d3408f95022bf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.601404 28802 ts_tablet_manager.cc:1929] T 7e82dc44779c451bb7e961e775672de1 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:19.601557 28802 log.cc:1199] T 7e82dc44779c451bb7e961e775672de1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/7e82dc44779c451bb7e961e775672de1
I20260618 13:59:19.601969 28802 ts_tablet_manager.cc:1950] T 7e82dc44779c451bb7e961e775672de1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:19.602223 28220 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 2efebe57e3d548fa8919221028142bb1
I20260618 13:59:19.602875 28220 catalog_manager.cc:6784] ReplaceTablet: tablet 2efebe57e3d548fa8919221028142bb1 deleted and replaced by tablet 717c38fbd9cb492b9547372e2160f84f
I20260618 13:59:19.602937 28663 tablet_service.cc:1511] Processing CreateTablet for tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 (DEFAULT_TABLE table=default.Table_e10ecf3a190148d0b17f97204f155353 [id=179292b8d0574f7bb5b6edb5dcad5cb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.603281 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fd3fbcf0d9745a1b3d3408f95022bf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.603777 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 7e82dc44779c451bb7e961e775672de1 (table default.Table_7a5b9ad4d9b44f358878aae89187c35a [id=74c1ef2bdbe342ff85c1d825b800a1cf]) successfully deleted
I20260618 13:59:19.604070 28802 tablet_replica.cc:333] T 734374fd92d346c4a9a38c9a3d687df2 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:19.604375 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet 2efebe57e3d548fa8919221028142bb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 717c38fbd9cb492b9547372e2160f84f) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:19.605418 28730 heartbeater.cc:646] Failed to heartbeat to 127.27.100.61:45967 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv got EOF from 127.27.100.61:45967 (error 108)
I20260618 13:59:19.606153 28661 tablet_service.cc:1511] Processing CreateTablet for tablet 587b82a6da8d4fe48b26070803831d9d (DEFAULT_TABLE table=default.Table_e7787f99590e428ca58ae371cdfeffd5 [id=262236d66ee24ad8a9e4e27b4b1c3ac1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.606379 28528 tablet_service.cc:1511] Processing CreateTablet for tablet 91c250dfbd72416d9d74f6cbb67b2bdf (DEFAULT_TABLE table=default.Table_3b5a67572c914054908f9bde6bb0ee09 [id=95f51150591e45e8a69bfacc9ac17f24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.606560 28661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 587b82a6da8d4fe48b26070803831d9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.606786 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91c250dfbd72416d9d74f6cbb67b2bdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.609012 28659 tablet_service.cc:1558] Processing DeleteTablet for tablet 2efebe57e3d548fa8919221028142bb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 717c38fbd9cb492b9547372e2160f84f) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.610771 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 587b82a6da8d4fe48b26070803831d9d (DEFAULT_TABLE table=default.Table_e7787f99590e428ca58ae371cdfeffd5 [id=262236d66ee24ad8a9e4e27b4b1c3ac1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.611166 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 587b82a6da8d4fe48b26070803831d9d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:19.611550 28485 connection.cc:573] client connection to 127.27.100.61:45967 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260618 13:59:19.612414 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 91c250dfbd72416d9d74f6cbb67b2bdf (DEFAULT_TABLE table=default.Table_3b5a67572c914054908f9bde6bb0ee09 [id=95f51150591e45e8a69bfacc9ac17f24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.612751 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91c250dfbd72416d9d74f6cbb67b2bdf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:19.613131 28458 heartbeater.cc:646] Failed to heartbeat to 127.27.100.61:45967 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv got EOF from 127.27.100.61:45967 (error 108)
I20260618 13:59:19.613560 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_479c1c1ed7d14546932db3cd2001df61"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:19.613699 28525 tablet_service.cc:1558] Processing DeleteTablet for tablet c1af8b68bd424b92ae7f5a57f8efb5e5 with delete_type TABLET_DATA_DELETED (replaced by tablet fc685eec02804dfabe2e25405b96def3) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:19.613900 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet 2efebe57e3d548fa8919221028142bb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 717c38fbd9cb492b9547372e2160f84f) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:19.614219 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_479c1c1ed7d14546932db3cd2001df61 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:19.614495 28596 heartbeater.cc:646] Failed to heartbeat to 127.27.100.61:45967 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260618 13:59:19.614548 28658 tablet_service.cc:1558] Processing DeleteTablet for tablet c1af8b68bd424b92ae7f5a57f8efb5e5 with delete_type TABLET_DATA_DELETED (replaced by tablet fc685eec02804dfabe2e25405b96def3) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.614848 28800 tablet_replica.cc:333] T 2efebe57e3d548fa8919221028142bb1 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:19.615103 28801 tablet_replica.cc:333] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:19.615310 28800 raft_consensus.cc:2243] T 2efebe57e3d548fa8919221028142bb1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:19.615442 28801 raft_consensus.cc:2243] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:19.615676 28800 raft_consensus.cc:2272] T 2efebe57e3d548fa8919221028142bb1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:19.615773 28801 raft_consensus.cc:2272] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:19.616492 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_1ccdff414ebc4079933d0edf079a2fdb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:19.616566 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_c8b7510465a8486c81d3da2179a9b563"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.617048 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ccdff414ebc4079933d0edf079a2fdb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:19.617132 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c8b7510465a8486c81d3da2179a9b563 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.617436 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_31b56ffcb8004359a6089d2f1e21aba7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.617908 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_31b56ffcb8004359a6089d2f1e21aba7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.619663 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_5224a46184b74b75a0379955524146f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.620095 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5224a46184b74b75a0379955524146f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.620216 28801 ts_tablet_manager.cc:1916] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:19.620277 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_1fd86f80c8fc4490aaadc1278d20df3a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.620996 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1fd86f80c8fc4490aaadc1278d20df3a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.622234 28800 ts_tablet_manager.cc:1916] T 2efebe57e3d548fa8919221028142bb1 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:19.623041 28799 tablet_bootstrap.cc:492] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:19.623209 28799 ts_tablet_manager.cc:1403] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.044s user 0.003s sys 0.000s
I20260618 13:59:19.623442 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_07cff5be28854452b80fdac31e7ecba3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.624006 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_07cff5be28854452b80fdac31e7ecba3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.624037 28799 raft_consensus.cc:359] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:19.624229 28799 raft_consensus.cc:385] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.624320 28799 raft_consensus.cc:740] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:19.624513 28799 consensus_queue.cc:260] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:19.624872 28216 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.100.61:45967 (error 108)
I20260618 13:59:19.624979 28799 ts_tablet_manager.cc:1434] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:19.625262 28799 tablet_bootstrap.cc:492] T 2efebe57e3d548fa8919221028142bb1 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:19.626025 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_bd146d3bdafe4c66b00a25d98887f1e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.626473 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd146d3bdafe4c66b00a25d98887f1e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.626498 28229 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_959239563d384ebdaf43cecfe9ce456d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.626991 28229 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_959239563d384ebdaf43cecfe9ce456d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.628587 28225 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_bc76ef57312a4ffda7ec3e4ba944ded2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.629022 28225 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc76ef57312a4ffda7ec3e4ba944ded2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.640389 28802 raft_consensus.cc:2243] T 734374fd92d346c4a9a38c9a3d687df2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:19.640697 28802 raft_consensus.cc:2272] T 734374fd92d346c4a9a38c9a3d687df2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:19.640867 28801 ts_tablet_manager.cc:1929] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:19.641036 28801 log.cc:1199] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/c1af8b68bd424b92ae7f5a57f8efb5e5
I20260618 13:59:19.641522 28801 ts_tablet_manager.cc:1950] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:19.643934 28799 tablet_bootstrap.cc:654] T 2efebe57e3d548fa8919221028142bb1 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.644490 28802 ts_tablet_manager.cc:1916] T 734374fd92d346c4a9a38c9a3d687df2 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:19.647863 28220 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_08cb9bd9bbb442648c47edbdd3a13ecd" } new_table_name: "default.Table_4cf6e1e03c724f42a0d15aed65142720" modify_external_catalogs: true
I20260618 13:59:19.649770 28798 tablet_bootstrap.cc:492] T 2efebe57e3d548fa8919221028142bb1 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:19.649930 28798 ts_tablet_manager.cc:1403] T 2efebe57e3d548fa8919221028142bb1 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.071s user 0.006s sys 0.011s
I20260618 13:59:19.649901 28224 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_a3eb0e4580814fc392f14934ec66ed3c" } new_table_name: "default.Table_34aad16c9cb043f3bca2d5eda89fdecb" modify_external_catalogs: true
I20260618 13:59:19.650090 28226 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_1c9a4bfc47d4424aab1b47c294cc1ff5" } new_table_name: "default.Table_58d6a901ec6a4741b232e4d1fe515cb5" modify_external_catalogs: true
I20260618 13:59:19.653728 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet c1af8b68bd424b92ae7f5a57f8efb5e5 with delete_type TABLET_DATA_DELETED (replaced by tablet fc685eec02804dfabe2e25405b96def3) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:19.653822 28798 raft_consensus.cc:359] T 2efebe57e3d548fa8919221028142bb1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.654026 28798 raft_consensus.cc:385] T 2efebe57e3d548fa8919221028142bb1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.654122 28798 raft_consensus.cc:740] T 2efebe57e3d548fa8919221028142bb1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:19.654851 28386 tablet_service.cc:1511] Processing CreateTablet for tablet 717c38fbd9cb492b9547372e2160f84f (DEFAULT_TABLE table=default.Table_fced244393ee4bd3ac9796af6863c9dc [id=3bcc877eddec4d00b3dc1760cc635b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.655316 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 717c38fbd9cb492b9547372e2160f84f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.656306 28528 tablet_service.cc:1511] Processing CreateTablet for tablet 717c38fbd9cb492b9547372e2160f84f (DEFAULT_TABLE table=default.Table_fced244393ee4bd3ac9796af6863c9dc [id=3bcc877eddec4d00b3dc1760cc635b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.656673 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 717c38fbd9cb492b9547372e2160f84f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.656667 28658 tablet_service.cc:1511] Processing CreateTablet for tablet 717c38fbd9cb492b9547372e2160f84f (DEFAULT_TABLE table=default.Table_fced244393ee4bd3ac9796af6863c9dc [id=3bcc877eddec4d00b3dc1760cc635b36]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:19.656968 28216 consensus_peers.cc:597] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 -> Peer a950ab7cd66b4cb0b29592aa692c2dd0 (127.27.100.61:45967): Couldn't send request to peer a950ab7cd66b4cb0b29592aa692c2dd0. Status: Network error: Client connection negotiation failed: client connection to 127.27.100.61:45967: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:19.657147 28658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 717c38fbd9cb492b9547372e2160f84f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.660733 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet c1af8b68bd424b92ae7f5a57f8efb5e5 (table default.Table_00e4e33ddb8948a79faa0174798b215f [id=aae6979440544fe984453a915e7c2e00]) successfully deleted
W20260618 13:59:19.661026 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 2efebe57e3d548fa8919221028142bb1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2efebe57e3d548fa8919221028142bb1 already in progress: creating tablet
I20260618 13:59:19.661907 28227 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_8dd5e514382c480eace10306357d5964" } new_table_name: "default.Table_94d9d315c21e4cfb9950d96262c9b6e3" modify_external_catalogs: true
I20260618 13:59:19.662905 28798 consensus_queue.cc:260] T 2efebe57e3d548fa8919221028142bb1 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.663448 28798 ts_tablet_manager.cc:1434] T 2efebe57e3d548fa8919221028142bb1 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260618 13:59:19.663753 28798 tablet_bootstrap.cc:492] T 7e82dc44779c451bb7e961e775672de1 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:19.665632 28798 tablet_bootstrap.cc:654] T 7e82dc44779c451bb7e961e775672de1 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.666002 28223 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_fced244393ee4bd3ac9796af6863c9dc" } new_table_name: "default.Table_d59cf7de23a34041aa4c0e2c4c0e4c83" modify_external_catalogs: true
I20260618 13:59:19.667821 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet 2efebe57e3d548fa8919221028142bb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 717c38fbd9cb492b9547372e2160f84f) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:19.668090 28801 tablet_replica.cc:333] T 2efebe57e3d548fa8919221028142bb1 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:19.668277 28801 raft_consensus.cc:2243] T 2efebe57e3d548fa8919221028142bb1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:19.668437 28801 raft_consensus.cc:2272] T 2efebe57e3d548fa8919221028142bb1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:19.669093 28799 tablet_bootstrap.cc:492] T 2efebe57e3d548fa8919221028142bb1 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:19.669262 28799 ts_tablet_manager.cc:1403] T 2efebe57e3d548fa8919221028142bb1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.044s user 0.005s sys 0.001s
W20260618 13:59:19.669699 28816 negotiation.cc:336] Failed RPC negotiation. Trace:
0618 13:59:19.613715 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.27.100.61:45967 (local address 127.27.100.1:38747)
0618 13:59:19.661978 (+ 48263us) negotiation.cc:107] Waiting for socket to connect
0618 13:59:19.663363 (+ 1385us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.100.61:45967: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":48112,"thread_start_us":48386,"threads_started":1}
I20260618 13:59:19.670125 28799 raft_consensus.cc:359] T 2efebe57e3d548fa8919221028142bb1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.670328 28799 raft_consensus.cc:385] T 2efebe57e3d548fa8919221028142bb1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.670403 28799 raft_consensus.cc:740] T 2efebe57e3d548fa8919221028142bb1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:19.671252 28801 ts_tablet_manager.cc:1916] T 2efebe57e3d548fa8919221028142bb1 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:19.672773 28815 negotiation.cc:336] Failed RPC negotiation. Trace:
0618 13:59:19.606019 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.27.100.61:45967 (local address 127.27.100.3:33429)
0618 13:59:19.667651 (+ 61632us) negotiation.cc:107] Waiting for socket to connect
0618 13:59:19.667850 (+ 199us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.100.61:45967: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":61511,"threads_started":1}
I20260618 13:59:19.674633 28799 consensus_queue.cc:260] T 2efebe57e3d548fa8919221028142bb1 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.676191 28802 ts_tablet_manager.cc:1929] T 734374fd92d346c4a9a38c9a3d687df2 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:19.676827 28802 log.cc:1199] T 734374fd92d346c4a9a38c9a3d687df2 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/734374fd92d346c4a9a38c9a3d687df2
I20260618 13:59:19.677338 28802 ts_tablet_manager.cc:1950] T 734374fd92d346c4a9a38c9a3d687df2 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:19.681631 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 734374fd92d346c4a9a38c9a3d687df2 (table default.Table_8dd5e514382c480eace10306357d5964 [id=a874707bba8d41e7b598d7ad4a3e21f3]) successfully deleted
I20260618 13:59:19.681800 28802 tablet_replica.cc:333] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
W20260618 13:59:19.681923 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 2efebe57e3d548fa8919221028142bb1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2efebe57e3d548fa8919221028142bb1 already in progress: creating tablet
I20260618 13:59:19.681978 28802 raft_consensus.cc:2243] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:19.682123 28802 raft_consensus.cc:2272] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:19.682870 28802 ts_tablet_manager.cc:1916] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:19.683638 28220 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f4447a3e87bd476e84f59dd4c4185b26 consensus state has no leader
I20260618 13:59:19.685993 28799 ts_tablet_manager.cc:1434] T 2efebe57e3d548fa8919221028142bb1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.017s user 0.000s sys 0.002s
I20260618 13:59:19.688102 28801 ts_tablet_manager.cc:1929] T 2efebe57e3d548fa8919221028142bb1 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:19.688268 28800 ts_tablet_manager.cc:1929] T 2efebe57e3d548fa8919221028142bb1 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:19.688580 28801 log.cc:1199] T 2efebe57e3d548fa8919221028142bb1 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/2efebe57e3d548fa8919221028142bb1
I20260618 13:59:19.688830 28802 ts_tablet_manager.cc:1929] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:19.688988 28802 log.cc:1199] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/c1af8b68bd424b92ae7f5a57f8efb5e5
I20260618 13:59:19.689214 28800 log.cc:1199] T 2efebe57e3d548fa8919221028142bb1 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/2efebe57e3d548fa8919221028142bb1
I20260618 13:59:19.689678 28802 ts_tablet_manager.cc:1950] T c1af8b68bd424b92ae7f5a57f8efb5e5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:19.689728 28800 ts_tablet_manager.cc:1950] T 2efebe57e3d548fa8919221028142bb1 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:19.689905 28799 tablet_bootstrap.cc:492] T 657635f877e24b76879843d61b55dd62 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:19.690104 28801 ts_tablet_manager.cc:1950] T 2efebe57e3d548fa8919221028142bb1 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:19.690680 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet c1af8b68bd424b92ae7f5a57f8efb5e5 (table default.Table_00e4e33ddb8948a79faa0174798b215f [id=aae6979440544fe984453a915e7c2e00]) successfully deleted
I20260618 13:59:19.691946 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 2efebe57e3d548fa8919221028142bb1 (table default.Table_fced244393ee4bd3ac9796af6863c9dc [id=3bcc877eddec4d00b3dc1760cc635b36]) successfully deleted
I20260618 13:59:19.692297 28800 tablet_replica.cc:333] T c1af8b68bd424b92ae7f5a57f8efb5e5 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:19.692386 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 2efebe57e3d548fa8919221028142bb1 (table default.Table_fced244393ee4bd3ac9796af6863c9dc [id=3bcc877eddec4d00b3dc1760cc635b36]) successfully deleted
I20260618 13:59:19.692507 28800 raft_consensus.cc:2243] T c1af8b68bd424b92ae7f5a57f8efb5e5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:19.692679 28800 raft_consensus.cc:2272] T c1af8b68bd424b92ae7f5a57f8efb5e5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:19.694321 28799 tablet_bootstrap.cc:654] T 657635f877e24b76879843d61b55dd62 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.698304 28528 tablet_service.cc:1511] Processing CreateTablet for tablet ee783d2c7e274158b3de2cc186f50903 (DEFAULT_TABLE table=default.Table_479c1c1ed7d14546932db3cd2001df61 [id=d8ef984c664f4a6598c70a48e9aff826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.698910 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee783d2c7e274158b3de2cc186f50903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.703334 28229 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_00e4e33ddb8948a79faa0174798b215f" } modify_external_catalogs: true
I20260618 13:59:19.703425 28220 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_3b5a67572c914054908f9bde6bb0ee09" } modify_external_catalogs: true
I20260618 13:59:19.703545 28229 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_00e4e33ddb8948a79faa0174798b215f" } modify_external_catalogs: true
I20260618 13:59:19.703704 28220 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_3b5a67572c914054908f9bde6bb0ee09" } modify_external_catalogs: true
W20260618 13:59:19.704983 28224 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 397c7abe26f4451298558d6a0875b677 consensus state has no leader
W20260618 13:59:19.705888 28226 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 59613383d2294f9ab6bce6321aad44bb consensus state has no leader
I20260618 13:59:19.706862 28226 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_c8b7510465a8486c81d3da2179a9b563" } modify_external_catalogs: true
I20260618 13:59:19.707105 28226 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_c8b7510465a8486c81d3da2179a9b563" } modify_external_catalogs: true
I20260618 13:59:19.708107 28799 tablet_bootstrap.cc:492] T 657635f877e24b76879843d61b55dd62 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:19.708375 28799 ts_tablet_manager.cc:1403] T 657635f877e24b76879843d61b55dd62 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.005s
I20260618 13:59:19.708997 28386 tablet_service.cc:1511] Processing CreateTablet for tablet ee783d2c7e274158b3de2cc186f50903 (DEFAULT_TABLE table=default.Table_479c1c1ed7d14546932db3cd2001df61 [id=d8ef984c664f4a6598c70a48e9aff826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.709012 28660 tablet_service.cc:1511] Processing CreateTablet for tablet ee783d2c7e274158b3de2cc186f50903 (DEFAULT_TABLE table=default.Table_479c1c1ed7d14546932db3cd2001df61 [id=d8ef984c664f4a6598c70a48e9aff826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.709465 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee783d2c7e274158b3de2cc186f50903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.709806 28799 raft_consensus.cc:359] T 657635f877e24b76879843d61b55dd62 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.710033 28799 raft_consensus.cc:385] T 657635f877e24b76879843d61b55dd62 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.710134 28799 raft_consensus.cc:740] T 657635f877e24b76879843d61b55dd62 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:19.710376 28797 tablet_bootstrap.cc:492] T 7e82dc44779c451bb7e961e775672de1 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:19.710374 28799 consensus_queue.cc:260] T 657635f877e24b76879843d61b55dd62 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.711328 28799 ts_tablet_manager.cc:1434] T 657635f877e24b76879843d61b55dd62 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:19.711627 28799 tablet_bootstrap.cc:492] T 68b3bb6a4dd3446288822b5424fe5122 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:19.713491 28227 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 68b3bb6a4dd3446288822b5424fe5122 consensus state has no leader
I20260618 13:59:19.710930 28797 ts_tablet_manager.cc:1403] T 7e82dc44779c451bb7e961e775672de1 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.144s user 0.002s sys 0.019s
W20260618 13:59:19.717757 28223 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 717c38fbd9cb492b9547372e2160f84f consensus state has no leader
I20260618 13:59:19.717810 28797 raft_consensus.cc:359] T 7e82dc44779c451bb7e961e775672de1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.718091 28797 raft_consensus.cc:385] T 7e82dc44779c451bb7e961e775672de1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.718182 28797 raft_consensus.cc:740] T 7e82dc44779c451bb7e961e775672de1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:19.718863 28797 consensus_queue.cc:260] T 7e82dc44779c451bb7e961e775672de1 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.719125 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 2efebe57e3d548fa8919221028142bb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 717c38fbd9cb492b9547372e2160f84f) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:19.719753 28229 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet fc685eec02804dfabe2e25405b96def3
I20260618 13:59:19.721029 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet fc685eec02804dfabe2e25405b96def3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:19.721212 28799 tablet_bootstrap.cc:654] T 68b3bb6a4dd3446288822b5424fe5122 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.722507 28802 tablet_replica.cc:333] T 2efebe57e3d548fa8919221028142bb1 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:19.722734 28802 raft_consensus.cc:2243] T 2efebe57e3d548fa8919221028142bb1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:19.722954 28802 raft_consensus.cc:2272] T 2efebe57e3d548fa8919221028142bb1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:19.723366 28220 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 91c250dfbd72416d9d74f6cbb67b2bdf
I20260618 13:59:19.723779 28802 ts_tablet_manager.cc:1916] T 2efebe57e3d548fa8919221028142bb1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:19.724372 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet 91c250dfbd72416d9d74f6cbb67b2bdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:19.725098 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 91c250dfbd72416d9d74f6cbb67b2bdf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91c250dfbd72416d9d74f6cbb67b2bdf already in progress: creating tablet
I20260618 13:59:19.725471 28226 catalog_manager.cc:5978] Not sending DeleteTablet requests; no consensus state for tablet a9baa12e1c864f7091836a5ab3ad010f
I20260618 13:59:19.725731 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet fc685eec02804dfabe2e25405b96def3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:19.724675 28387 tablet_service.cc:1558] Processing DeleteTablet for tablet 91c250dfbd72416d9d74f6cbb67b2bdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:19.724851 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 657635f877e24b76879843d61b55dd62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:19.726410 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet fc685eec02804dfabe2e25405b96def3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc685eec02804dfabe2e25405b96def3 already in progress: creating tablet
I20260618 13:59:19.695024 28800 ts_tablet_manager.cc:1916] T c1af8b68bd424b92ae7f5a57f8efb5e5 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:19.729214 28661 tablet_service.cc:1558] Processing DeleteTablet for tablet 91c250dfbd72416d9d74f6cbb67b2bdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.729393 28797 ts_tablet_manager.cc:1434] T 7e82dc44779c451bb7e961e775672de1 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20260618 13:59:19.729477 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet fc685eec02804dfabe2e25405b96def3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
W20260618 13:59:19.730113 28817 negotiation.cc:336] Failed RPC negotiation. Trace:
0618 13:59:19.618726 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.27.100.61:45967 (local address 127.27.100.2:41679)
0618 13:59:19.693269 (+ 74543us) negotiation.cc:107] Waiting for socket to connect
0618 13:59:19.693468 (+ 199us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.100.61:45967: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":74445,"thread_start_us":74304,"threads_started":1}
I20260618 13:59:19.730515 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee783d2c7e274158b3de2cc186f50903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.734663 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e82dc44779c451bb7e961e775672de1 with delete_type TABLET_DATA_DELETED (replaced by tablet 657635f877e24b76879843d61b55dd62) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.735915 28226 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_e7787f99590e428ca58ae371cdfeffd5" } modify_external_catalogs: true
I20260618 13:59:19.736033 28220 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_07cff5be28854452b80fdac31e7ecba3" } modify_external_catalogs: true
I20260618 13:59:19.736258 28220 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_07cff5be28854452b80fdac31e7ecba3" } modify_external_catalogs: true
I20260618 13:59:19.736248 28226 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_e7787f99590e428ca58ae371cdfeffd5" } modify_external_catalogs: true
I20260618 13:59:19.738912 28802 ts_tablet_manager.cc:1929] T 2efebe57e3d548fa8919221028142bb1 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:19.739112 28802 log.cc:1199] T 2efebe57e3d548fa8919221028142bb1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/2efebe57e3d548fa8919221028142bb1
I20260618 13:59:19.739601 28802 ts_tablet_manager.cc:1950] T 2efebe57e3d548fa8919221028142bb1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:19.742925 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 2efebe57e3d548fa8919221028142bb1 (table default.Table_d59cf7de23a34041aa4c0e2c4c0e4c83 [id=3bcc877eddec4d00b3dc1760cc635b36]) successfully deleted
W20260618 13:59:19.743237 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet fc685eec02804dfabe2e25405b96def3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc685eec02804dfabe2e25405b96def3 already in progress: creating tablet
W20260618 13:59:19.744457 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 91c250dfbd72416d9d74f6cbb67b2bdf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91c250dfbd72416d9d74f6cbb67b2bdf already in progress: creating tablet
I20260618 13:59:19.744782 28802 tablet_replica.cc:333] T 657635f877e24b76879843d61b55dd62 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:19.745023 28802 raft_consensus.cc:2243] T 657635f877e24b76879843d61b55dd62 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:19.745282 28802 raft_consensus.cc:2272] T 657635f877e24b76879843d61b55dd62 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:19.745929 28797 tablet_bootstrap.cc:492] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:19.746381 28802 ts_tablet_manager.cc:1916] T 657635f877e24b76879843d61b55dd62 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:19.750142 28383 tablet_service.cc:1511] Processing CreateTablet for tablet d5cb3137659a4481840fe04174ef682d (DEFAULT_TABLE table=default.Table_bc76ef57312a4ffda7ec3e4ba944ded2 [id=38bde7a301814aa285e4bd30bcc527aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.750707 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5cb3137659a4481840fe04174ef682d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.753712 28797 tablet_bootstrap.cc:654] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.754786 28391 tablet_service.cc:1511] Processing CreateTablet for tablet 570d36a27b7d417989ad72885d38ff63 (DEFAULT_TABLE table=default.Table_959239563d384ebdaf43cecfe9ce456d [id=e38a333ca62d42518ab53a9ac17df2b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.755182 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 570d36a27b7d417989ad72885d38ff63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.756628 28388 tablet_service.cc:1511] Processing CreateTablet for tablet b312fbf800434ecb840dd10347efe6e8 (DEFAULT_TABLE table=default.Table_1ccdff414ebc4079933d0edf079a2fdb [id=7e70d6cbf42e437da0c1bb47eee6633c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.757097 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b312fbf800434ecb840dd10347efe6e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.758703 28387 tablet_service.cc:1511] Processing CreateTablet for tablet 102a500268574e428cb2d45e52849f57 (DEFAULT_TABLE table=default.Table_bd146d3bdafe4c66b00a25d98887f1e2 [id=76865706d6a54ec198dd205b6ad61920]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.759147 28387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102a500268574e428cb2d45e52849f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.757094 28802 ts_tablet_manager.cc:1929] T 657635f877e24b76879843d61b55dd62 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:19.760093 28802 log.cc:1199] T 657635f877e24b76879843d61b55dd62 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/657635f877e24b76879843d61b55dd62
I20260618 13:59:19.760541 28802 ts_tablet_manager.cc:1950] T 657635f877e24b76879843d61b55dd62 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:19.755098 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 8c9d89fa75174a969ded07e0596db99b (DEFAULT_TABLE table=default.Table_07cff5be28854452b80fdac31e7ecba3 [id=b024b7c54de64df0a4047d8f0381a5c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.762959 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c9d89fa75174a969ded07e0596db99b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.764123 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 657635f877e24b76879843d61b55dd62 (table default.Table_7a5b9ad4d9b44f358878aae89187c35a [id=74c1ef2bdbe342ff85c1d825b800a1cf]) successfully deleted
I20260618 13:59:19.765483 28390 tablet_service.cc:1511] Processing CreateTablet for tablet a2ca1e22c38e49d793d413e9eaaac06f (DEFAULT_TABLE table=default.Table_5224a46184b74b75a0379955524146f3 [id=f071d496d807405cb0f3b022757b85e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.765830 28390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2ca1e22c38e49d793d413e9eaaac06f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.768088 28226 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 587b82a6da8d4fe48b26070803831d9d
I20260618 13:59:19.770290 28226 master_service.cc:960] ReplaceTablet: received request to replace tablet 07a4e95bd6ae47beac34fd36d6e956e3 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:19.770689 28657 tablet_service.cc:1511] Processing CreateTablet for tablet a2ca1e22c38e49d793d413e9eaaac06f (DEFAULT_TABLE table=default.Table_5224a46184b74b75a0379955524146f3 [id=f071d496d807405cb0f3b022757b85e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.770741 28388 tablet_service.cc:1511] Processing CreateTablet for tablet 144aa8984a44410fb3f90dc99e9b2274 (DEFAULT_TABLE table=default.Table_31b56ffcb8004359a6089d2f1e21aba7 [id=c2109218729e47e0b1057c9802e312a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.770907 28659 tablet_service.cc:1511] Processing CreateTablet for tablet 144aa8984a44410fb3f90dc99e9b2274 (DEFAULT_TABLE table=default.Table_31b56ffcb8004359a6089d2f1e21aba7 [id=c2109218729e47e0b1057c9802e312a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.771090 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2ca1e22c38e49d793d413e9eaaac06f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.771281 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 144aa8984a44410fb3f90dc99e9b2274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.771413 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 144aa8984a44410fb3f90dc99e9b2274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.771525 28661 tablet_service.cc:1511] Processing CreateTablet for tablet 570d36a27b7d417989ad72885d38ff63 (DEFAULT_TABLE table=default.Table_959239563d384ebdaf43cecfe9ce456d [id=e38a333ca62d42518ab53a9ac17df2b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.771857 28661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 570d36a27b7d417989ad72885d38ff63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.774612 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet 587b82a6da8d4fe48b26070803831d9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.775923 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 587b82a6da8d4fe48b26070803831d9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:19.775976 28527 tablet_service.cc:1511] Processing CreateTablet for tablet 102a500268574e428cb2d45e52849f57 (DEFAULT_TABLE table=default.Table_bd146d3bdafe4c66b00a25d98887f1e2 [id=76865706d6a54ec198dd205b6ad61920]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.776345 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102a500268574e428cb2d45e52849f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.777735 28798 tablet_bootstrap.cc:492] T 7e82dc44779c451bb7e961e775672de1 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:19.777940 28798 ts_tablet_manager.cc:1403] T 7e82dc44779c451bb7e961e775672de1 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.114s user 0.005s sys 0.027s
I20260618 13:59:19.778721 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 570d36a27b7d417989ad72885d38ff63 (DEFAULT_TABLE table=default.Table_959239563d384ebdaf43cecfe9ce456d [id=e38a333ca62d42518ab53a9ac17df2b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.778721 28663 tablet_service.cc:1511] Processing CreateTablet for tablet b312fbf800434ecb840dd10347efe6e8 (DEFAULT_TABLE table=default.Table_1ccdff414ebc4079933d0edf079a2fdb [id=7e70d6cbf42e437da0c1bb47eee6633c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.778934 28798 raft_consensus.cc:359] T 7e82dc44779c451bb7e961e775672de1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.779098 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b312fbf800434ecb840dd10347efe6e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.779199 28798 raft_consensus.cc:385] T 7e82dc44779c451bb7e961e775672de1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.779282 28798 raft_consensus.cc:740] T 7e82dc44779c451bb7e961e775672de1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:19.779384 28524 tablet_service.cc:1511] Processing CreateTablet for tablet 8c9d89fa75174a969ded07e0596db99b (DEFAULT_TABLE table=default.Table_07cff5be28854452b80fdac31e7ecba3 [id=b024b7c54de64df0a4047d8f0381a5c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.779556 28798 consensus_queue.cc:260] T 7e82dc44779c451bb7e961e775672de1 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.779985 28525 tablet_service.cc:1511] Processing CreateTablet for tablet b312fbf800434ecb840dd10347efe6e8 (DEFAULT_TABLE table=default.Table_1ccdff414ebc4079933d0edf079a2fdb [id=7e70d6cbf42e437da0c1bb47eee6633c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.780117 28798 ts_tablet_manager.cc:1434] T 7e82dc44779c451bb7e961e775672de1 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260618 13:59:19.780218 28800 ts_tablet_manager.cc:1929] T c1af8b68bd424b92ae7f5a57f8efb5e5 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:19.780367 28800 log.cc:1199] T c1af8b68bd424b92ae7f5a57f8efb5e5 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/c1af8b68bd424b92ae7f5a57f8efb5e5
I20260618 13:59:19.780496 28529 tablet_service.cc:1511] Processing CreateTablet for tablet 144aa8984a44410fb3f90dc99e9b2274 (DEFAULT_TABLE table=default.Table_31b56ffcb8004359a6089d2f1e21aba7 [id=c2109218729e47e0b1057c9802e312a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.780854 28800 ts_tablet_manager.cc:1950] T c1af8b68bd424b92ae7f5a57f8efb5e5 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:19.781087 28523 tablet_service.cc:1511] Processing CreateTablet for tablet a2ca1e22c38e49d793d413e9eaaac06f (DEFAULT_TABLE table=default.Table_5224a46184b74b75a0379955524146f3 [id=f071d496d807405cb0f3b022757b85e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.779088 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 570d36a27b7d417989ad72885d38ff63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.783681 28799 tablet_bootstrap.cc:492] T 68b3bb6a4dd3446288822b5424fe5122 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:19.783883 28799 ts_tablet_manager.cc:1403] T 68b3bb6a4dd3446288822b5424fe5122 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.072s user 0.013s sys 0.019s
I20260618 13:59:19.784893 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet 587b82a6da8d4fe48b26070803831d9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:19.785367 28658 tablet_service.cc:1511] Processing CreateTablet for tablet 8c9d89fa75174a969ded07e0596db99b (DEFAULT_TABLE table=default.Table_07cff5be28854452b80fdac31e7ecba3 [id=b024b7c54de64df0a4047d8f0381a5c8]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:19.785408 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 91c250dfbd72416d9d74f6cbb67b2bdf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91c250dfbd72416d9d74f6cbb67b2bdf already in progress: creating tablet
W20260618 13:59:19.785590 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 587b82a6da8d4fe48b26070803831d9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 587b82a6da8d4fe48b26070803831d9d already in progress: creating tablet
W20260618 13:59:19.785687 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet fc685eec02804dfabe2e25405b96def3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc685eec02804dfabe2e25405b96def3 already in progress: creating tablet
W20260618 13:59:19.785914 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 587b82a6da8d4fe48b26070803831d9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 587b82a6da8d4fe48b26070803831d9d already in progress: creating tablet
I20260618 13:59:19.785842 28799 raft_consensus.cc:359] T 68b3bb6a4dd3446288822b5424fe5122 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.785914 28662 tablet_service.cc:1511] Processing CreateTablet for tablet 102a500268574e428cb2d45e52849f57 (DEFAULT_TABLE table=default.Table_bd146d3bdafe4c66b00a25d98887f1e2 [id=76865706d6a54ec198dd205b6ad61920]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.786067 28799 raft_consensus.cc:385] T 68b3bb6a4dd3446288822b5424fe5122 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.786155 28799 raft_consensus.cc:740] T 68b3bb6a4dd3446288822b5424fe5122 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:19.786386 28228 master_service.cc:960] ReplaceTablet: received request to replace tablet 68b3bb6a4dd3446288822b5424fe5122 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:19.786468 28820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7637285570574a6dbcef1de6f84ecac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:19.786643 28820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: This master's current role is: LEADER
I20260618 13:59:19.786518 28799 consensus_queue.cc:260] T 68b3bb6a4dd3446288822b5424fe5122 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.789513 28799 ts_tablet_manager.cc:1434] T 68b3bb6a4dd3446288822b5424fe5122 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260618 13:59:19.789763 28798 tablet_bootstrap.cc:492] T 734374fd92d346c4a9a38c9a3d687df2 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:19.789882 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 144aa8984a44410fb3f90dc99e9b2274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.790048 28799 tablet_bootstrap.cc:492] T fc685eec02804dfabe2e25405b96def3 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:19.791980 28799 tablet_bootstrap.cc:654] T fc685eec02804dfabe2e25405b96def3 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.795065 28658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c9d89fa75174a969ded07e0596db99b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.795290 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c9d89fa75174a969ded07e0596db99b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.795938 28220 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 8c9d89fa75174a969ded07e0596db99b
I20260618 13:59:19.798544 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b312fbf800434ecb840dd10347efe6e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.799795 28226 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 07a4e95bd6ae47beac34fd36d6e956e3
I20260618 13:59:19.800248 28226 catalog_manager.cc:6784] ReplaceTablet: tablet 07a4e95bd6ae47beac34fd36d6e956e3 deleted and replaced by tablet eeffc96b9b97441d9301e4919d0047e1
I20260618 13:59:19.800908 28228 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 68b3bb6a4dd3446288822b5424fe5122
I20260618 13:59:19.801340 28228 catalog_manager.cc:6784] ReplaceTablet: tablet 68b3bb6a4dd3446288822b5424fe5122 deleted and replaced by tablet 85a3b27d914a412498919a3f65e20522
I20260618 13:59:19.802057 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102a500268574e428cb2d45e52849f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.802033 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet 68b3bb6a4dd3446288822b5424fe5122 with delete_type TABLET_DATA_DELETED (replaced by tablet 85a3b27d914a412498919a3f65e20522) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:19.803010 28802 tablet_replica.cc:333] T 68b3bb6a4dd3446288822b5424fe5122 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:19.803174 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2ca1e22c38e49d793d413e9eaaac06f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.803192 28802 raft_consensus.cc:2243] T 68b3bb6a4dd3446288822b5424fe5122 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:19.803378 28802 raft_consensus.cc:2272] T 68b3bb6a4dd3446288822b5424fe5122 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:19.804579 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet 07a4e95bd6ae47beac34fd36d6e956e3 with delete_type TABLET_DATA_DELETED (replaced by tablet eeffc96b9b97441d9301e4919d0047e1) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.804621 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 68b3bb6a4dd3446288822b5424fe5122 with delete_type TABLET_DATA_DELETED (replaced by tablet 85a3b27d914a412498919a3f65e20522) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.810444 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet 07a4e95bd6ae47beac34fd36d6e956e3 with delete_type TABLET_DATA_DELETED (replaced by tablet eeffc96b9b97441d9301e4919d0047e1) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:19.810827 28229 master_service.cc:960] ReplaceTablet: received request to replace tablet f7027a4138784d029255fc89c73104a3 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:19.812943 28388 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c9d89fa75174a969ded07e0596db99b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:19.814049 28799 tablet_bootstrap.cc:492] T fc685eec02804dfabe2e25405b96def3 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:19.814213 28799 ts_tablet_manager.cc:1403] T fc685eec02804dfabe2e25405b96def3 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.008s
I20260618 13:59:19.815153 28799 raft_consensus.cc:359] T fc685eec02804dfabe2e25405b96def3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.815352 28799 raft_consensus.cc:385] T fc685eec02804dfabe2e25405b96def3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.815464 28799 raft_consensus.cc:740] T fc685eec02804dfabe2e25405b96def3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:19.815752 28799 consensus_queue.cc:260] T fc685eec02804dfabe2e25405b96def3 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.813179 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e82dc44779c451bb7e961e775672de1 with delete_type TABLET_DATA_DELETED (replaced by tablet 657635f877e24b76879843d61b55dd62) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:19.816445 28799 ts_tablet_manager.cc:1434] T fc685eec02804dfabe2e25405b96def3 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 13:59:19.816956 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c9d89fa75174a969ded07e0596db99b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:19.813771 28528 tablet_service.cc:1511] Processing CreateTablet for tablet d5cb3137659a4481840fe04174ef682d (DEFAULT_TABLE table=default.Table_bc76ef57312a4ffda7ec3e4ba944ded2 [id=38bde7a301814aa285e4bd30bcc527aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.818018 28801 tablet_replica.cc:333] T 7e82dc44779c451bb7e961e775672de1 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:19.818212 28801 raft_consensus.cc:2243] T 7e82dc44779c451bb7e961e775672de1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:19.819350 28802 ts_tablet_manager.cc:1916] T 68b3bb6a4dd3446288822b5424fe5122 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:19.821785 28801 raft_consensus.cc:2272] T 7e82dc44779c451bb7e961e775672de1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:19.824568 28802 ts_tablet_manager.cc:1929] T 68b3bb6a4dd3446288822b5424fe5122 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:19.824676 28802 log.cc:1199] T 68b3bb6a4dd3446288822b5424fe5122 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/68b3bb6a4dd3446288822b5424fe5122
I20260618 13:59:19.824777 28660 tablet_service.cc:1511] Processing CreateTablet for tablet d5cb3137659a4481840fe04174ef682d (DEFAULT_TABLE table=default.Table_bc76ef57312a4ffda7ec3e4ba944ded2 [id=38bde7a301814aa285e4bd30bcc527aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.825114 28802 ts_tablet_manager.cc:1950] T 68b3bb6a4dd3446288822b5424fe5122 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:19.825218 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5cb3137659a4481840fe04174ef682d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.825685 28658 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c9d89fa75174a969ded07e0596db99b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.825796 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet 07a4e95bd6ae47beac34fd36d6e956e3 with delete_type TABLET_DATA_DELETED (replaced by tablet eeffc96b9b97441d9301e4919d0047e1) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:19.825979 28800 tablet_replica.cc:333] T 7e82dc44779c451bb7e961e775672de1 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:19.826009 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 68b3bb6a4dd3446288822b5424fe5122 with delete_type TABLET_DATA_DELETED (replaced by tablet 85a3b27d914a412498919a3f65e20522) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:19.826135 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5cb3137659a4481840fe04174ef682d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.826239 28800 raft_consensus.cc:2243] T 7e82dc44779c451bb7e961e775672de1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:19.826541 28800 raft_consensus.cc:2272] T 7e82dc44779c451bb7e961e775672de1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:19.827603 28800 ts_tablet_manager.cc:1916] T 7e82dc44779c451bb7e961e775672de1 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:19.831087 28801 ts_tablet_manager.cc:1916] T 7e82dc44779c451bb7e961e775672de1 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:19.839692 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 15bbf81e3fbf4927b822afa0635909eb (DEFAULT_TABLE table=default.Table_1fd86f80c8fc4490aaadc1278d20df3a [id=e76fba0e75f74f8a9d0a94ff9d7f7b53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.842720 28229 master_service.cc:960] ReplaceTablet: received request to replace tablet 59613383d2294f9ab6bce6321aad44bb from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:19.844856 28799 tablet_bootstrap.cc:492] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:19.845927 28800 ts_tablet_manager.cc:1929] T 7e82dc44779c451bb7e961e775672de1 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:19.847620 28799 tablet_bootstrap.cc:654] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.848290 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15bbf81e3fbf4927b822afa0635909eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.848452 28800 log.cc:1199] T 7e82dc44779c451bb7e961e775672de1 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/7e82dc44779c451bb7e961e775672de1
I20260618 13:59:19.848914 28800 ts_tablet_manager.cc:1950] T 7e82dc44779c451bb7e961e775672de1 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:19.849524 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 68b3bb6a4dd3446288822b5424fe5122 (table default.Table_94d9d315c21e4cfb9950d96262c9b6e3 [id=a874707bba8d41e7b598d7ad4a3e21f3]) successfully deleted
W20260618 13:59:19.849822 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 07a4e95bd6ae47beac34fd36d6e956e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07a4e95bd6ae47beac34fd36d6e956e3 already in progress: creating tablet
I20260618 13:59:19.850008 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 7e82dc44779c451bb7e961e775672de1 (table default.Table_7a5b9ad4d9b44f358878aae89187c35a [id=74c1ef2bdbe342ff85c1d825b800a1cf]) successfully deleted
W20260618 13:59:19.850132 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 8c9d89fa75174a969ded07e0596db99b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8c9d89fa75174a969ded07e0596db99b already in progress: creating tablet
I20260618 13:59:19.850462 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet fc685eec02804dfabe2e25405b96def3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:19.850737 28802 tablet_replica.cc:333] T fc685eec02804dfabe2e25405b96def3 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:19.851015 28802 raft_consensus.cc:2243] T fc685eec02804dfabe2e25405b96def3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:19.851263 28802 raft_consensus.cc:2272] T fc685eec02804dfabe2e25405b96def3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:19.851662 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 587b82a6da8d4fe48b26070803831d9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 587b82a6da8d4fe48b26070803831d9d already in progress: creating tablet
I20260618 13:59:19.852906 28802 ts_tablet_manager.cc:1916] T fc685eec02804dfabe2e25405b96def3 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:19.856746 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 85a3b27d914a412498919a3f65e20522 (DEFAULT_TABLE table=default.Table_94d9d315c21e4cfb9950d96262c9b6e3 [id=a874707bba8d41e7b598d7ad4a3e21f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.857208 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85a3b27d914a412498919a3f65e20522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.859354 28798 tablet_bootstrap.cc:654] T 734374fd92d346c4a9a38c9a3d687df2 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.865177 28663 tablet_service.cc:1511] Processing CreateTablet for tablet 85a3b27d914a412498919a3f65e20522 (DEFAULT_TABLE table=default.Table_94d9d315c21e4cfb9950d96262c9b6e3 [id=a874707bba8d41e7b598d7ad4a3e21f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.867323 28801 ts_tablet_manager.cc:1929] T 7e82dc44779c451bb7e961e775672de1 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:19.867457 28801 log.cc:1199] T 7e82dc44779c451bb7e961e775672de1 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/7e82dc44779c451bb7e961e775672de1
I20260618 13:59:19.867743 28802 ts_tablet_manager.cc:1929] T fc685eec02804dfabe2e25405b96def3 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:19.867857 28801 ts_tablet_manager.cc:1950] T 7e82dc44779c451bb7e961e775672de1 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:19.867947 28802 log.cc:1199] T fc685eec02804dfabe2e25405b96def3 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/fc685eec02804dfabe2e25405b96def3
I20260618 13:59:19.868468 28802 ts_tablet_manager.cc:1950] T fc685eec02804dfabe2e25405b96def3 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:19.868526 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_a9703f9d3b54453691c354131f824e22"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.869143 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a9703f9d3b54453691c354131f824e22 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.869908 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet fc685eec02804dfabe2e25405b96def3 (table default.Table_00e4e33ddb8948a79faa0174798b215f [id=aae6979440544fe984453a915e7c2e00]) successfully deleted
I20260618 13:59:19.870675 28388 tablet_service.cc:1511] Processing CreateTablet for tablet 15bbf81e3fbf4927b822afa0635909eb (DEFAULT_TABLE table=default.Table_1fd86f80c8fc4490aaadc1278d20df3a [id=e76fba0e75f74f8a9d0a94ff9d7f7b53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.871266 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15bbf81e3fbf4927b822afa0635909eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.872349 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 15bbf81e3fbf4927b822afa0635909eb (DEFAULT_TABLE table=default.Table_1fd86f80c8fc4490aaadc1278d20df3a [id=e76fba0e75f74f8a9d0a94ff9d7f7b53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.872803 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15bbf81e3fbf4927b822afa0635909eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.872833 28798 tablet_bootstrap.cc:492] T 734374fd92d346c4a9a38c9a3d687df2 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:19.872989 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85a3b27d914a412498919a3f65e20522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.872995 28798 ts_tablet_manager.cc:1403] T 734374fd92d346c4a9a38c9a3d687df2 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.091s user 0.002s sys 0.002s
I20260618 13:59:19.873883 28798 raft_consensus.cc:359] T 734374fd92d346c4a9a38c9a3d687df2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.874082 28798 raft_consensus.cc:385] T 734374fd92d346c4a9a38c9a3d687df2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.874166 28798 raft_consensus.cc:740] T 734374fd92d346c4a9a38c9a3d687df2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:19.874466 28798 consensus_queue.cc:260] T 734374fd92d346c4a9a38c9a3d687df2 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.875984 28797 tablet_bootstrap.cc:492] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:19.876161 28797 ts_tablet_manager.cc:1403] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.130s user 0.025s sys 0.000s
I20260618 13:59:19.878014 28797 raft_consensus.cc:359] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.878216 28797 raft_consensus.cc:385] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.878325 28797 raft_consensus.cc:740] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:19.878607 28797 consensus_queue.cc:260] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.878693 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_48dff8bdaa4e430b8850f8d688195bcf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:19.878832 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_6b0a93aa5c0d4d1e9806a4d6e52491c6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:19.879134 28797 ts_tablet_manager.cc:1434] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260618 13:59:19.879297 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b0a93aa5c0d4d1e9806a4d6e52491c6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.879642 28798 ts_tablet_manager.cc:1434] T 734374fd92d346c4a9a38c9a3d687df2 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260618 13:59:19.881624 28798 tablet_bootstrap.cc:492] T 73f048d041fe4d2e8d944e446cbf099c P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
W20260618 13:59:19.882210 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_48dff8bdaa4e430b8850f8d688195bcf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.883598 28797 tablet_bootstrap.cc:492] T 73f048d041fe4d2e8d944e446cbf099c P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:19.884693 28798 tablet_bootstrap.cc:654] T 73f048d041fe4d2e8d944e446cbf099c P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.885131 28799 tablet_bootstrap.cc:492] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:19.885381 28799 ts_tablet_manager.cc:1403] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.041s user 0.013s sys 0.011s
I20260618 13:59:19.886252 28799 raft_consensus.cc:359] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.886544 28799 raft_consensus.cc:385] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.886720 28799 raft_consensus.cc:740] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:19.887058 28799 consensus_queue.cc:260] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.889024 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_823368bbdf8242739a6022f65280f5a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:19.889281 28225 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_c272009caa8640008b0cc2ef4592d87e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:19.889487 28222 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 397c7abe26f4451298558d6a0875b677 (table default.Table_34aad16c9cb043f3bca2d5eda89fdecb [id=ce780967c12a412a8d7f0bef2f9c00e3]). Expected version 3 got 0
W20260618 13:59:19.889622 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_823368bbdf8242739a6022f65280f5a2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:19.890003 28225 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c272009caa8640008b0cc2ef4592d87e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.891240 28224 master_service.cc:960] ReplaceTablet: received request to replace tablet 73f048d041fe4d2e8d944e446cbf099c from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:19.892859 28799 ts_tablet_manager.cc:1434] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260618 13:59:19.894233 28799 tablet_bootstrap.cc:492] T 59613383d2294f9ab6bce6321aad44bb P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:19.895998 28223 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 397c7abe26f4451298558d6a0875b677 (table default.Table_34aad16c9cb043f3bca2d5eda89fdecb [id=ce780967c12a412a8d7f0bef2f9c00e3]). Expected version 3 got 0
I20260618 13:59:19.896385 28799 tablet_bootstrap.cc:654] T 59613383d2294f9ab6bce6321aad44bb P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.898216 28388 tablet_service.cc:1511] Processing CreateTablet for tablet eeffc96b9b97441d9301e4919d0047e1 (DEFAULT_TABLE table=default.Table_b964eadba5974835bbacd9c0b049aaf3 [id=972bc2c124ac459da06ec4093c5b564b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.898360 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_b35e2c7c7707422d93364bf9be3bab3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:19.898826 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeffc96b9b97441d9301e4919d0047e1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:19.899037 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b35e2c7c7707422d93364bf9be3bab3f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.902169 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_cd191bffa0274a44be88412fadf7faa3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.902894 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cd191bffa0274a44be88412fadf7faa3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.905975 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_e0177aee52ca4680a5786ad73020ad75"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:19.906404 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_bf7cbaec816a4c04a225ff31f57123ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:19.906769 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e0177aee52ca4680a5786ad73020ad75 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:19.907126 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bf7cbaec816a4c04a225ff31f57123ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.913281 28806 raft_consensus.cc:493] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:19.913533 28806 raft_consensus.cc:515] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.914788 28797 tablet_bootstrap.cc:654] T 73f048d041fe4d2e8d944e446cbf099c P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.917363 28806 leader_election.cc:290] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:19.919871 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet 734374fd92d346c4a9a38c9a3d687df2 with delete_type TABLET_DATA_DELETED (replaced by tablet 68b3bb6a4dd3446288822b5424fe5122) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:19.920151 28801 tablet_replica.cc:333] T 734374fd92d346c4a9a38c9a3d687df2 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:19.920342 28801 raft_consensus.cc:2243] T 734374fd92d346c4a9a38c9a3d687df2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:19.920527 28801 raft_consensus.cc:2272] T 734374fd92d346c4a9a38c9a3d687df2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:19.920711 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 07a4e95bd6ae47beac34fd36d6e956e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07a4e95bd6ae47beac34fd36d6e956e3 already in progress: creating tablet
I20260618 13:59:19.921305 28660 tablet_service.cc:1511] Processing CreateTablet for tablet eeffc96b9b97441d9301e4919d0047e1 (DEFAULT_TABLE table=default.Table_b964eadba5974835bbacd9c0b049aaf3 [id=972bc2c124ac459da06ec4093c5b564b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.921713 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeffc96b9b97441d9301e4919d0047e1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:19.922840 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 68b3bb6a4dd3446288822b5424fe5122 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68b3bb6a4dd3446288822b5424fe5122 already in progress: creating tablet
W20260618 13:59:19.923032 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 8c9d89fa75174a969ded07e0596db99b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8c9d89fa75174a969ded07e0596db99b already in progress: creating tablet
I20260618 13:59:19.923352 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 7e82dc44779c451bb7e961e775672de1 (table default.Table_7a5b9ad4d9b44f358878aae89187c35a [id=74c1ef2bdbe342ff85c1d825b800a1cf]) successfully deleted
W20260618 13:59:19.923607 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 8c9d89fa75174a969ded07e0596db99b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8c9d89fa75174a969ded07e0596db99b already in progress: creating tablet
W20260618 13:59:19.923808 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 07a4e95bd6ae47beac34fd36d6e956e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07a4e95bd6ae47beac34fd36d6e956e3 already in progress: creating tablet
W20260618 13:59:19.923978 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 68b3bb6a4dd3446288822b5424fe5122 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68b3bb6a4dd3446288822b5424fe5122 already in progress: creating tablet
I20260618 13:59:19.926950 28229 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 59613383d2294f9ab6bce6321aad44bb
I20260618 13:59:19.927915 28229 catalog_manager.cc:6784] ReplaceTablet: tablet 59613383d2294f9ab6bce6321aad44bb deleted and replaced by tablet 29bf263d20934ed6a6efb4b215252c00
I20260618 13:59:19.927196 28525 tablet_service.cc:1511] Processing CreateTablet for tablet eeffc96b9b97441d9301e4919d0047e1 (DEFAULT_TABLE table=default.Table_b964eadba5974835bbacd9c0b049aaf3 [id=972bc2c124ac459da06ec4093c5b564b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.928489 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeffc96b9b97441d9301e4919d0047e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.929711 28229 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_959239563d384ebdaf43cecfe9ce456d" } new_table_name: "default.Table_f5ba3d11da184f25b976a8bc7c54cce1" modify_external_catalogs: true
I20260618 13:59:19.934273 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 59613383d2294f9ab6bce6321aad44bb with delete_type TABLET_DATA_DELETED (replaced by tablet 29bf263d20934ed6a6efb4b215252c00) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:19.935089 28801 ts_tablet_manager.cc:1916] T 734374fd92d346c4a9a38c9a3d687df2 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:19.935132 28797 tablet_bootstrap.cc:492] T 73f048d041fe4d2e8d944e446cbf099c P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
W20260618 13:59:19.935191 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 59613383d2294f9ab6bce6321aad44bb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59613383d2294f9ab6bce6321aad44bb already in progress: creating tablet
I20260618 13:59:19.935283 28797 ts_tablet_manager.cc:1403] T 73f048d041fe4d2e8d944e446cbf099c P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.052s user 0.002s sys 0.002s
I20260618 13:59:19.936053 28797 raft_consensus.cc:359] T 73f048d041fe4d2e8d944e446cbf099c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.936246 28797 raft_consensus.cc:385] T 73f048d041fe4d2e8d944e446cbf099c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.936308 28797 raft_consensus.cc:740] T 73f048d041fe4d2e8d944e446cbf099c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:19.936533 28797 consensus_queue.cc:260] T 73f048d041fe4d2e8d944e446cbf099c P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.936981 28797 ts_tablet_manager.cc:1434] T 73f048d041fe4d2e8d944e446cbf099c P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:19.938237 28797 tablet_bootstrap.cc:492] T fc685eec02804dfabe2e25405b96def3 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:19.927585 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 85a3b27d914a412498919a3f65e20522 (DEFAULT_TABLE table=default.Table_94d9d315c21e4cfb9950d96262c9b6e3 [id=a874707bba8d41e7b598d7ad4a3e21f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:19.939565 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85a3b27d914a412498919a3f65e20522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:19.940011 28797 tablet_bootstrap.cc:654] T fc685eec02804dfabe2e25405b96def3 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.946362 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet 59613383d2294f9ab6bce6321aad44bb with delete_type TABLET_DATA_DELETED (replaced by tablet 29bf263d20934ed6a6efb4b215252c00) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:19.947234 28658 tablet_service.cc:1558] Processing DeleteTablet for tablet 59613383d2294f9ab6bce6321aad44bb with delete_type TABLET_DATA_DELETED (replaced by tablet 29bf263d20934ed6a6efb4b215252c00) from {username='slave'} at 127.0.0.1:43976
W20260618 13:59:19.948285 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 59613383d2294f9ab6bce6321aad44bb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59613383d2294f9ab6bce6321aad44bb already in progress: creating tablet
I20260618 13:59:19.950795 28797 tablet_bootstrap.cc:492] T fc685eec02804dfabe2e25405b96def3 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:19.950961 28797 ts_tablet_manager.cc:1403] T fc685eec02804dfabe2e25405b96def3 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260618 13:59:19.951900 28797 raft_consensus.cc:359] T fc685eec02804dfabe2e25405b96def3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.952127 28797 raft_consensus.cc:385] T fc685eec02804dfabe2e25405b96def3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:19.952231 28797 raft_consensus.cc:740] T fc685eec02804dfabe2e25405b96def3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:19.952487 28797 consensus_queue.cc:260] T fc685eec02804dfabe2e25405b96def3 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:19.953193 28797 ts_tablet_manager.cc:1434] T fc685eec02804dfabe2e25405b96def3 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:19.953507 28797 tablet_bootstrap.cc:492] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:19.955417 28797 tablet_bootstrap.cc:654] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:19.957278 28801 ts_tablet_manager.cc:1929] T 734374fd92d346c4a9a38c9a3d687df2 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:19.957455 28801 log.cc:1199] T 734374fd92d346c4a9a38c9a3d687df2 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/734374fd92d346c4a9a38c9a3d687df2
I20260618 13:59:19.957854 28801 ts_tablet_manager.cc:1950] T 734374fd92d346c4a9a38c9a3d687df2 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:19.961246 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_e4cb8c97170c465980e9eeb3f6e4f88c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:19.961937 28228 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_bc76ef57312a4ffda7ec3e4ba944ded2" } new_table_name: "default.Table_c97dcabd4c2b45a18f445a099ec7ace9" modify_external_catalogs: true
I20260618 13:59:19.963490 28220 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_1cdc3c58c3964fd6a6f8137d22460d61" } new_table_name: "default.Table_4ac6ae42a7f94b79abb170afc27cd978" modify_external_catalogs: true
W20260618 13:59:19.964159 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e4cb8c97170c465980e9eeb3f6e4f88c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:19.966706 28226 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_b964eadba5974835bbacd9c0b049aaf3" } new_table_name: "default.Table_f0607bf6529e4f209e00f49908420141" modify_external_catalogs: true
I20260618 13:59:19.968358 28225 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_d59cf7de23a34041aa4c0e2c4c0e4c83" } new_table_name: "default.Table_e395544c03754cc9a2cb0514a31326e3" modify_external_catalogs: true
I20260618 13:59:19.970084 28221 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_5224a46184b74b75a0379955524146f3" } modify_external_catalogs: true
I20260618 13:59:19.970274 28221 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_5224a46184b74b75a0379955524146f3" } modify_external_catalogs: true
I20260618 13:59:19.971048 28223 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_94d9d315c21e4cfb9950d96262c9b6e3" } modify_external_catalogs: true
I20260618 13:59:19.971202 28223 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_94d9d315c21e4cfb9950d96262c9b6e3" } modify_external_catalogs: true
I20260618 13:59:19.975656 28222 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_bd146d3bdafe4c66b00a25d98887f1e2" } modify_external_catalogs: true
I20260618 13:59:19.975839 28222 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_bd146d3bdafe4c66b00a25d98887f1e2" } modify_external_catalogs: true
I20260618 13:59:19.983585 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet 73f048d041fe4d2e8d944e446cbf099c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.984015 28800 tablet_replica.cc:333] T 73f048d041fe4d2e8d944e446cbf099c P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:19.984247 28800 raft_consensus.cc:2243] T 73f048d041fe4d2e8d944e446cbf099c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:19.984452 28800 raft_consensus.cc:2272] T 73f048d041fe4d2e8d944e446cbf099c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:19.985312 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet fc685eec02804dfabe2e25405b96def3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:19.985368 28800 ts_tablet_manager.cc:1916] T 73f048d041fe4d2e8d944e446cbf099c P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:19.992161 28800 ts_tablet_manager.cc:1929] T 73f048d041fe4d2e8d944e446cbf099c P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:19.992408 28800 log.cc:1199] T 73f048d041fe4d2e8d944e446cbf099c P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/73f048d041fe4d2e8d944e446cbf099c
I20260618 13:59:19.992864 28800 ts_tablet_manager.cc:1950] T 73f048d041fe4d2e8d944e446cbf099c P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:19.994051 28800 tablet_replica.cc:333] T fc685eec02804dfabe2e25405b96def3 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:19.994318 28800 raft_consensus.cc:2243] T fc685eec02804dfabe2e25405b96def3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:19.994585 28800 raft_consensus.cc:2272] T fc685eec02804dfabe2e25405b96def3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:19.994529 28224 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_1ccdff414ebc4079933d0edf079a2fdb" } modify_external_catalogs: true
I20260618 13:59:19.994779 28224 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_1ccdff414ebc4079933d0edf079a2fdb" } modify_external_catalogs: true
I20260618 13:59:19.995532 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 734374fd92d346c4a9a38c9a3d687df2 (table default.Table_94d9d315c21e4cfb9950d96262c9b6e3 [id=a874707bba8d41e7b598d7ad4a3e21f3]) successfully deleted
W20260618 13:59:19.995808 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 59613383d2294f9ab6bce6321aad44bb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59613383d2294f9ab6bce6321aad44bb already in progress: creating tablet
I20260618 13:59:19.996106 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 73f048d041fe4d2e8d944e446cbf099c (table default.Table_15b7dcd63a7747e2b69f95da678dbc61 [id=474f40a5bbf846d2bd37eeb3acbe8bda]) successfully deleted
I20260618 13:59:19.996129 28663 tablet_service.cc:1558] Processing DeleteTablet for tablet 734374fd92d346c4a9a38c9a3d687df2 with delete_type TABLET_DATA_DELETED (replaced by tablet 68b3bb6a4dd3446288822b5424fe5122) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.000101 28800 ts_tablet_manager.cc:1916] T fc685eec02804dfabe2e25405b96def3 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:20.002405 28228 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d5cb3137659a4481840fe04174ef682d consensus state has no leader
W20260618 13:59:20.002835 28220 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a6189fed26fb471c9724fd594054a58c consensus state has no leader
W20260618 13:59:20.003727 28226 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet eeffc96b9b97441d9301e4919d0047e1 consensus state has no leader
W20260618 13:59:20.004207 28229 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 570d36a27b7d417989ad72885d38ff63 consensus state has no leader
I20260618 13:59:20.010265 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 29bf263d20934ed6a6efb4b215252c00 (DEFAULT_TABLE table=default.Table_58d6a901ec6a4741b232e4d1fe515cb5 [id=7923680bde144e3dbb78c922d09540b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.010939 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29bf263d20934ed6a6efb4b215252c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.018638 28525 tablet_service.cc:1511] Processing CreateTablet for tablet 29bf263d20934ed6a6efb4b215252c00 (DEFAULT_TABLE table=default.Table_58d6a901ec6a4741b232e4d1fe515cb5 [id=7923680bde144e3dbb78c922d09540b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.018702 28229 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_c272009caa8640008b0cc2ef4592d87e" } modify_external_catalogs: true
I20260618 13:59:20.018894 28229 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_c272009caa8640008b0cc2ef4592d87e" } modify_external_catalogs: true
I20260618 13:59:20.019241 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29bf263d20934ed6a6efb4b215252c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.025868 28663 tablet_service.cc:1511] Processing CreateTablet for tablet 29bf263d20934ed6a6efb4b215252c00 (DEFAULT_TABLE table=default.Table_58d6a901ec6a4741b232e4d1fe515cb5 [id=7923680bde144e3dbb78c922d09540b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.026358 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29bf263d20934ed6a6efb4b215252c00. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:20.026808 28225 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 717c38fbd9cb492b9547372e2160f84f consensus state has no leader
I20260618 13:59:20.028056 28222 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 102a500268574e428cb2d45e52849f57
I20260618 13:59:20.029153 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 102a500268574e428cb2d45e52849f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:20.030011 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 102a500268574e428cb2d45e52849f57 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 102a500268574e428cb2d45e52849f57 already in progress: creating tablet
I20260618 13:59:20.030821 28223 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 85a3b27d914a412498919a3f65e20522
I20260618 13:59:20.031927 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 85a3b27d914a412498919a3f65e20522 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:20.032636 28221 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet a2ca1e22c38e49d793d413e9eaaac06f
W20260618 13:59:20.032753 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 85a3b27d914a412498919a3f65e20522 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85a3b27d914a412498919a3f65e20522 already in progress: creating tablet
I20260618 13:59:20.033535 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet a2ca1e22c38e49d793d413e9eaaac06f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:20.034205 28224 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet b312fbf800434ecb840dd10347efe6e8
W20260618 13:59:20.034281 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet a2ca1e22c38e49d793d413e9eaaac06f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a2ca1e22c38e49d793d413e9eaaac06f already in progress: creating tablet
I20260618 13:59:20.035181 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet b312fbf800434ecb840dd10347efe6e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:20.035952 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet b312fbf800434ecb840dd10347efe6e8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b312fbf800434ecb840dd10347efe6e8 already in progress: creating tablet
I20260618 13:59:20.036882 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet 102a500268574e428cb2d45e52849f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.037796 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet 85a3b27d914a412498919a3f65e20522 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.038672 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet a2ca1e22c38e49d793d413e9eaaac06f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.038810 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet 85a3b27d914a412498919a3f65e20522 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.039650 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet b312fbf800434ecb840dd10347efe6e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.039705 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet a2ca1e22c38e49d793d413e9eaaac06f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.040133 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet 102a500268574e428cb2d45e52849f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.040208 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet b312fbf800434ecb840dd10347efe6e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.045893 28229 catalog_manager.cc:5978] Not sending DeleteTablet requests; no consensus state for tablet f85136382a984e6db686b195cb976423
W20260618 13:59:20.053352 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 85a3b27d914a412498919a3f65e20522 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85a3b27d914a412498919a3f65e20522 already in progress: creating tablet
I20260618 13:59:20.054580 28798 tablet_bootstrap.cc:492] T 73f048d041fe4d2e8d944e446cbf099c P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:20.054602 28797 tablet_bootstrap.cc:492] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:20.054775 28797 ts_tablet_manager.cc:1403] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.101s user 0.001s sys 0.003s
W20260618 13:59:20.055488 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet a2ca1e22c38e49d793d413e9eaaac06f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a2ca1e22c38e49d793d413e9eaaac06f already in progress: creating tablet
I20260618 13:59:20.055792 28798 ts_tablet_manager.cc:1403] T 73f048d041fe4d2e8d944e446cbf099c P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.174s user 0.052s sys 0.082s
I20260618 13:59:20.055675 28797 raft_consensus.cc:359] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.055905 28797 raft_consensus.cc:385] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.055989 28797 raft_consensus.cc:740] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.056242 28797 consensus_queue.cc:260] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:20.056578 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet b312fbf800434ecb840dd10347efe6e8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b312fbf800434ecb840dd10347efe6e8 already in progress: creating tablet
W20260618 13:59:20.056869 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 102a500268574e428cb2d45e52849f57 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 102a500268574e428cb2d45e52849f57 already in progress: creating tablet
I20260618 13:59:20.057463 28798 raft_consensus.cc:359] T 73f048d041fe4d2e8d944e446cbf099c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.058054 28229 master_service.cc:960] ReplaceTablet: received request to replace tablet ee783d2c7e274158b3de2cc186f50903 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:20.058521 28221 master_service.cc:960] ReplaceTablet: received request to replace tablet ee783d2c7e274158b3de2cc186f50903 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:20.059105 28799 tablet_bootstrap.cc:492] T 59613383d2294f9ab6bce6321aad44bb P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:20.059249 28799 ts_tablet_manager.cc:1403] T 59613383d2294f9ab6bce6321aad44bb P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.165s user 0.039s sys 0.088s
I20260618 13:59:20.059057 28798 raft_consensus.cc:385] T 73f048d041fe4d2e8d944e446cbf099c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.059541 28223 master_service.cc:960] ReplaceTablet: received request to replace tablet a6189fed26fb471c9724fd594054a58c from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:20.059772 28798 raft_consensus.cc:740] T 73f048d041fe4d2e8d944e446cbf099c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.060642 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet 657635f877e24b76879843d61b55dd62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.060604 28798 consensus_queue.cc:260] T 73f048d041fe4d2e8d944e446cbf099c P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.061156 28798 ts_tablet_manager.cc:1434] T 73f048d041fe4d2e8d944e446cbf099c P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260618 13:59:20.061579 28797 ts_tablet_manager.cc:1434] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260618 13:59:20.066205 28797 tablet_bootstrap.cc:492] T 68b3bb6a4dd3446288822b5424fe5122 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:20.059928 28799 raft_consensus.cc:359] T 59613383d2294f9ab6bce6321aad44bb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.066710 28799 raft_consensus.cc:385] T 59613383d2294f9ab6bce6321aad44bb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.066819 28799 raft_consensus.cc:740] T 59613383d2294f9ab6bce6321aad44bb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.067045 28799 consensus_queue.cc:260] T 59613383d2294f9ab6bce6321aad44bb P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.068053 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 5332b6bb76ea419086341738fd135b24 (DEFAULT_TABLE table=default.Table_e4cb8c97170c465980e9eeb3f6e4f88c [id=116d7007e8754aab8d37956bcb596f13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.068462 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5332b6bb76ea419086341738fd135b24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.072834 28798 tablet_bootstrap.cc:492] T 68b3bb6a4dd3446288822b5424fe5122 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:20.074602 28798 tablet_bootstrap.cc:654] T 68b3bb6a4dd3446288822b5424fe5122 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.075598 28524 tablet_service.cc:1511] Processing CreateTablet for tablet d6e3060118e844559803a649224d7474 (DEFAULT_TABLE table=default.Table_b35e2c7c7707422d93364bf9be3bab3f [id=625859c44f3246eaacad14ba1f027d9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.075685 28387 tablet_service.cc:1558] Processing DeleteTablet for tablet 59613383d2294f9ab6bce6321aad44bb with delete_type TABLET_DATA_DELETED (replaced by tablet 29bf263d20934ed6a6efb4b215252c00) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:20.075632 28388 tablet_service.cc:1511] Processing CreateTablet for tablet d6e3060118e844559803a649224d7474 (DEFAULT_TABLE table=default.Table_b35e2c7c7707422d93364bf9be3bab3f [id=625859c44f3246eaacad14ba1f027d9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.075963 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6e3060118e844559803a649224d7474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.076001 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6e3060118e844559803a649224d7474. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:20.076663 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 59613383d2294f9ab6bce6321aad44bb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59613383d2294f9ab6bce6321aad44bb already in progress: creating tablet
I20260618 13:59:20.077176 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 059716cec7f94ea6ad8989fecc6f832a (DEFAULT_TABLE table=default.Table_e0177aee52ca4680a5786ad73020ad75 [id=591e2be369c347e883141ed55927044d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.077556 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 059716cec7f94ea6ad8989fecc6f832a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.078357 28387 tablet_service.cc:1511] Processing CreateTablet for tablet b8389b77e6b8407db38e07e85f7509b1 (DEFAULT_TABLE table=default.Table_a9703f9d3b54453691c354131f824e22 [id=f298b441bd9b444b9edd832be0adb03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.078881 28387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8389b77e6b8407db38e07e85f7509b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.078835 28528 tablet_service.cc:1511] Processing CreateTablet for tablet 1d23b5d2d67945a6ab9591969acb0d27 (DEFAULT_TABLE table=default.Table_48dff8bdaa4e430b8850f8d688195bcf [id=fa4cd457fb154247acbfc842852eef04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.078851 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 7e5517e820ae47de810ff817f5382839 (DEFAULT_TABLE table=default.Table_cd191bffa0274a44be88412fadf7faa3 [id=49576dabdecf41aca1b15ff3fc0c113c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.079200 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d23b5d2d67945a6ab9591969acb0d27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.079234 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e5517e820ae47de810ff817f5382839. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.082353 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet 73f048d041fe4d2e8d944e446cbf099c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.082698 28390 tablet_service.cc:1511] Processing CreateTablet for tablet 1d23b5d2d67945a6ab9591969acb0d27 (DEFAULT_TABLE table=default.Table_48dff8bdaa4e430b8850f8d688195bcf [id=fa4cd457fb154247acbfc842852eef04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.082891 28527 tablet_service.cc:1511] Processing CreateTablet for tablet 7e5517e820ae47de810ff817f5382839 (DEFAULT_TABLE table=default.Table_cd191bffa0274a44be88412fadf7faa3 [id=49576dabdecf41aca1b15ff3fc0c113c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.083093 28390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d23b5d2d67945a6ab9591969acb0d27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.083240 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e5517e820ae47de810ff817f5382839. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.089707 28526 tablet_service.cc:1511] Processing CreateTablet for tablet b8389b77e6b8407db38e07e85f7509b1 (DEFAULT_TABLE table=default.Table_a9703f9d3b54453691c354131f824e22 [id=f298b441bd9b444b9edd832be0adb03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.090082 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8389b77e6b8407db38e07e85f7509b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.092711 28522 tablet_service.cc:1511] Processing CreateTablet for tablet 18fb8adfc8ee40e18a1da5f978317124 (DEFAULT_TABLE table=default.Table_bf7cbaec816a4c04a225ff31f57123ed [id=f34f77924d464341afa518830ba7bbbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.093397 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18fb8adfc8ee40e18a1da5f978317124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.094764 28520 tablet_service.cc:1511] Processing CreateTablet for tablet 31b328882af549a78535cdd55c6ea09c (DEFAULT_TABLE table=default.Table_823368bbdf8242739a6022f65280f5a2 [id=28f3530b169a44eb9c262a718c8ea6d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.095352 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31b328882af549a78535cdd55c6ea09c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.095800 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 31b328882af549a78535cdd55c6ea09c (DEFAULT_TABLE table=default.Table_823368bbdf8242739a6022f65280f5a2 [id=28f3530b169a44eb9c262a718c8ea6d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.096177 28799 ts_tablet_manager.cc:1434] T 59613383d2294f9ab6bce6321aad44bb P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260618 13:59:20.096515 28799 tablet_bootstrap.cc:492] T 07a4e95bd6ae47beac34fd36d6e956e3 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:20.096804 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31b328882af549a78535cdd55c6ea09c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.099259 28529 tablet_service.cc:1511] Processing CreateTablet for tablet 411b0c333ae7450f9ada90afee553615 (DEFAULT_TABLE table=default.Table_6b0a93aa5c0d4d1e9806a4d6e52491c6 [id=cd89063e7d294d3fbb3b58358e781c90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.099757 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 411b0c333ae7450f9ada90afee553615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.107399 28391 tablet_service.cc:1511] Processing CreateTablet for tablet 411b0c333ae7450f9ada90afee553615 (DEFAULT_TABLE table=default.Table_6b0a93aa5c0d4d1e9806a4d6e52491c6 [id=cd89063e7d294d3fbb3b58358e781c90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.107904 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 411b0c333ae7450f9ada90afee553615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.075240 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 059716cec7f94ea6ad8989fecc6f832a (DEFAULT_TABLE table=default.Table_e0177aee52ca4680a5786ad73020ad75 [id=591e2be369c347e883141ed55927044d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.109865 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 059716cec7f94ea6ad8989fecc6f832a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.112775 28799 tablet_bootstrap.cc:654] T 07a4e95bd6ae47beac34fd36d6e956e3 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.113159 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet 59613383d2294f9ab6bce6321aad44bb with delete_type TABLET_DATA_DELETED (replaced by tablet 29bf263d20934ed6a6efb4b215252c00) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:20.113583 28802 tablet_replica.cc:333] T 59613383d2294f9ab6bce6321aad44bb P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:20.115382 28802 raft_consensus.cc:2243] T 59613383d2294f9ab6bce6321aad44bb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.116271 28223 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet a6189fed26fb471c9724fd594054a58c
I20260618 13:59:20.116780 28223 catalog_manager.cc:6784] ReplaceTablet: tablet a6189fed26fb471c9724fd594054a58c deleted and replaced by tablet 4ebed81a21b14db48ecb69299e0956cd
I20260618 13:59:20.118204 28229 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet ee783d2c7e274158b3de2cc186f50903
I20260618 13:59:20.118481 28802 raft_consensus.cc:2272] T 59613383d2294f9ab6bce6321aad44bb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.118674 28229 catalog_manager.cc:6784] ReplaceTablet: tablet ee783d2c7e274158b3de2cc186f50903 deleted and replaced by tablet 311ba4dd83984f379c8f4f8274bc26c1
I20260618 13:59:20.119920 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet a6189fed26fb471c9724fd594054a58c with delete_type TABLET_DATA_DELETED (replaced by tablet 4ebed81a21b14db48ecb69299e0956cd) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.120043 28801 tablet_replica.cc:333] T 73f048d041fe4d2e8d944e446cbf099c P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:20.120226 28801 raft_consensus.cc:2243] T 73f048d041fe4d2e8d944e446cbf099c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.120296 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet ee783d2c7e274158b3de2cc186f50903 with delete_type TABLET_DATA_DELETED (replaced by tablet 311ba4dd83984f379c8f4f8274bc26c1) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.120427 28801 raft_consensus.cc:2272] T 73f048d041fe4d2e8d944e446cbf099c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.121841 28387 tablet_service.cc:1511] Processing CreateTablet for tablet 18fb8adfc8ee40e18a1da5f978317124 (DEFAULT_TABLE table=default.Table_bf7cbaec816a4c04a225ff31f57123ed [id=f34f77924d464341afa518830ba7bbbe]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:20.121605 28459 tablet.cc:2406] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 13:59:20.126498 28388 tablet_service.cc:1558] Processing DeleteTablet for tablet a6189fed26fb471c9724fd594054a58c with delete_type TABLET_DATA_DELETED (replaced by tablet 4ebed81a21b14db48ecb69299e0956cd) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:20.126724 28221 master_service.cc:960] ReplaceTablet: received request to replace tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:20.127283 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet ee783d2c7e274158b3de2cc186f50903 with delete_type TABLET_DATA_DELETED (replaced by tablet 311ba4dd83984f379c8f4f8274bc26c1) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:20.127676 28802 ts_tablet_manager.cc:1916] T 59613383d2294f9ab6bce6321aad44bb P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.128281 28801 ts_tablet_manager.cc:1916] T 73f048d041fe4d2e8d944e446cbf099c P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.136512 28801 ts_tablet_manager.cc:1929] T 73f048d041fe4d2e8d944e446cbf099c P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.137051 28807 raft_consensus.cc:493] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:20.137401 28387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18fb8adfc8ee40e18a1da5f978317124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.137282 28807 raft_consensus.cc:515] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.139503 28807 leader_election.cc:290] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:20.141583 28801 log.cc:1199] T 73f048d041fe4d2e8d944e446cbf099c P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/73f048d041fe4d2e8d944e446cbf099c
I20260618 13:59:20.142131 28801 ts_tablet_manager.cc:1950] T 73f048d041fe4d2e8d944e446cbf099c P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:20.143157 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 73f048d041fe4d2e8d944e446cbf099c (table default.Table_15b7dcd63a7747e2b69f95da678dbc61 [id=474f40a5bbf846d2bd37eeb3acbe8bda]) successfully deleted
W20260618 13:59:20.143824 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet a6189fed26fb471c9724fd594054a58c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a6189fed26fb471c9724fd594054a58c already in progress: creating tablet
W20260618 13:59:20.150631 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet ee783d2c7e274158b3de2cc186f50903 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee783d2c7e274158b3de2cc186f50903 already in progress: creating tablet
I20260618 13:59:20.151450 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 311ba4dd83984f379c8f4f8274bc26c1 (DEFAULT_TABLE table=default.Table_479c1c1ed7d14546932db3cd2001df61 [id=d8ef984c664f4a6598c70a48e9aff826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.151887 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 311ba4dd83984f379c8f4f8274bc26c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.153017 28529 tablet_service.cc:1511] Processing CreateTablet for tablet 4ebed81a21b14db48ecb69299e0956cd (DEFAULT_TABLE table=default.Table_4ac6ae42a7f94b79abb170afc27cd978 [id=979317e20c204b109fb05c4128c0e2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.153350 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ebed81a21b14db48ecb69299e0956cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.155701 28386 tablet_service.cc:1511] Processing CreateTablet for tablet 311ba4dd83984f379c8f4f8274bc26c1 (DEFAULT_TABLE table=default.Table_479c1c1ed7d14546932db3cd2001df61 [id=d8ef984c664f4a6598c70a48e9aff826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.156107 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 311ba4dd83984f379c8f4f8274bc26c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.159917 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 5332b6bb76ea419086341738fd135b24 (DEFAULT_TABLE table=default.Table_e4cb8c97170c465980e9eeb3f6e4f88c [id=116d7007e8754aab8d37956bcb596f13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.160334 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "397c7abe26f4451298558d6a0875b677" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:20.161244 28221 master_service.cc:960] ReplaceTablet: received request to replace tablet ee783d2c7e274158b3de2cc186f50903 from {username='slave'} at 127.0.0.1:45138
W20260618 13:59:20.161445 28346 leader_election.cc:343] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:20.162526 28525 tablet_service.cc:1511] Processing CreateTablet for tablet 5332b6bb76ea419086341738fd135b24 (DEFAULT_TABLE table=default.Table_e4cb8c97170c465980e9eeb3f6e4f88c [id=116d7007e8754aab8d37956bcb596f13]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:20.162117 28618 leader_election.cc:343] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Not found: Tablet not found: 734374fd92d346c4a9a38c9a3d687df2
I20260618 13:59:20.162914 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5332b6bb76ea419086341738fd135b24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.162920 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 4ebed81a21b14db48ecb69299e0956cd (DEFAULT_TABLE table=default.Table_4ac6ae42a7f94b79abb170afc27cd978 [id=979317e20c204b109fb05c4128c0e2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.163257 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ebed81a21b14db48ecb69299e0956cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.163664 28661 tablet_service.cc:1511] Processing CreateTablet for tablet b8389b77e6b8407db38e07e85f7509b1 (DEFAULT_TABLE table=default.Table_a9703f9d3b54453691c354131f824e22 [id=f298b441bd9b444b9edd832be0adb03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.165059 28658 tablet_service.cc:1511] Processing CreateTablet for tablet 059716cec7f94ea6ad8989fecc6f832a (DEFAULT_TABLE table=default.Table_e0177aee52ca4680a5786ad73020ad75 [id=591e2be369c347e883141ed55927044d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.161072 28655 tablet_service.cc:1511] Processing CreateTablet for tablet 18fb8adfc8ee40e18a1da5f978317124 (DEFAULT_TABLE table=default.Table_bf7cbaec816a4c04a225ff31f57123ed [id=f34f77924d464341afa518830ba7bbbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.165721 28802 ts_tablet_manager.cc:1929] T 59613383d2294f9ab6bce6321aad44bb P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.165830 28802 log.cc:1199] T 59613383d2294f9ab6bce6321aad44bb P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/59613383d2294f9ab6bce6321aad44bb
I20260618 13:59:20.166826 28799 tablet_bootstrap.cc:492] T 07a4e95bd6ae47beac34fd36d6e956e3 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:20.167124 28799 ts_tablet_manager.cc:1403] T 07a4e95bd6ae47beac34fd36d6e956e3 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.071s user 0.004s sys 0.004s
W20260618 13:59:20.167171 28618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 68997.5us
I20260618 13:59:20.167799 28799 raft_consensus.cc:359] T 07a4e95bd6ae47beac34fd36d6e956e3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.169668 28659 tablet_service.cc:1511] Processing CreateTablet for tablet 1d23b5d2d67945a6ab9591969acb0d27 (DEFAULT_TABLE table=default.Table_48dff8bdaa4e430b8850f8d688195bcf [id=fa4cd457fb154247acbfc842852eef04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.170606 28799 raft_consensus.cc:385] T 07a4e95bd6ae47beac34fd36d6e956e3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.170691 28799 raft_consensus.cc:740] T 07a4e95bd6ae47beac34fd36d6e956e3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.171032 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5332b6bb76ea419086341738fd135b24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.170967 28799 consensus_queue.cc:260] T 07a4e95bd6ae47beac34fd36d6e956e3 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.171226 28657 tablet_service.cc:1511] Processing CreateTablet for tablet 7e5517e820ae47de810ff817f5382839 (DEFAULT_TABLE table=default.Table_cd191bffa0274a44be88412fadf7faa3 [id=49576dabdecf41aca1b15ff3fc0c113c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.171525 28799 ts_tablet_manager.cc:1434] T 07a4e95bd6ae47beac34fd36d6e956e3 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260618 13:59:20.171556 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e5517e820ae47de810ff817f5382839. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.171847 28799 tablet_bootstrap.cc:492] T a6189fed26fb471c9724fd594054a58c P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:20.173070 28802 ts_tablet_manager.cc:1950] T 59613383d2294f9ab6bce6321aad44bb P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:20.173897 28799 tablet_bootstrap.cc:654] T a6189fed26fb471c9724fd594054a58c P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.174507 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet ee783d2c7e274158b3de2cc186f50903 with delete_type TABLET_DATA_DELETED (replaced by tablet 311ba4dd83984f379c8f4f8274bc26c1) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.174685 28662 tablet_service.cc:1511] Processing CreateTablet for tablet d6e3060118e844559803a649224d7474 (DEFAULT_TABLE table=default.Table_b35e2c7c7707422d93364bf9be3bab3f [id=625859c44f3246eaacad14ba1f027d9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.175005 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6e3060118e844559803a649224d7474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.177469 28798 tablet_bootstrap.cc:492] T 68b3bb6a4dd3446288822b5424fe5122 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:20.177603 28798 ts_tablet_manager.cc:1403] T 68b3bb6a4dd3446288822b5424fe5122 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.105s user 0.032s sys 0.021s
I20260618 13:59:20.178375 28798 raft_consensus.cc:359] T 68b3bb6a4dd3446288822b5424fe5122 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.178505 28656 tablet_service.cc:1511] Processing CreateTablet for tablet 31b328882af549a78535cdd55c6ea09c (DEFAULT_TABLE table=default.Table_823368bbdf8242739a6022f65280f5a2 [id=28f3530b169a44eb9c262a718c8ea6d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.178588 28798 raft_consensus.cc:385] T 68b3bb6a4dd3446288822b5424fe5122 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.178669 28798 raft_consensus.cc:740] T 68b3bb6a4dd3446288822b5424fe5122 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.178941 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31b328882af549a78535cdd55c6ea09c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.178889 28798 consensus_queue.cc:260] T 68b3bb6a4dd3446288822b5424fe5122 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.179404 28798 ts_tablet_manager.cc:1434] T 68b3bb6a4dd3446288822b5424fe5122 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:20.179685 28798 tablet_bootstrap.cc:492] T fc685eec02804dfabe2e25405b96def3 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:20.181318 28798 tablet_bootstrap.cc:654] T fc685eec02804dfabe2e25405b96def3 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.182798 28651 tablet_service.cc:1511] Processing CreateTablet for tablet 4ebed81a21b14db48ecb69299e0956cd (DEFAULT_TABLE table=default.Table_4ac6ae42a7f94b79abb170afc27cd978 [id=979317e20c204b109fb05c4128c0e2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.183295 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ebed81a21b14db48ecb69299e0956cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.186498 28650 tablet_service.cc:1511] Processing CreateTablet for tablet 311ba4dd83984f379c8f4f8274bc26c1 (DEFAULT_TABLE table=default.Table_479c1c1ed7d14546932db3cd2001df61 [id=d8ef984c664f4a6598c70a48e9aff826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.187062 28650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 311ba4dd83984f379c8f4f8274bc26c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.188253 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 59613383d2294f9ab6bce6321aad44bb (table default.Table_58d6a901ec6a4741b232e4d1fe515cb5 [id=7923680bde144e3dbb78c922d09540b9]) successfully deleted
I20260618 13:59:20.190551 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet a6189fed26fb471c9724fd594054a58c with delete_type TABLET_DATA_DELETED (replaced by tablet 4ebed81a21b14db48ecb69299e0956cd) from {username='slave'} at 127.0.0.1:43976
W20260618 13:59:20.195152 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet a6189fed26fb471c9724fd594054a58c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a6189fed26fb471c9724fd594054a58c already in progress: creating tablet
I20260618 13:59:20.196794 28661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8389b77e6b8407db38e07e85f7509b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.197510 28654 tablet_service.cc:1511] Processing CreateTablet for tablet 411b0c333ae7450f9ada90afee553615 (DEFAULT_TABLE table=default.Table_6b0a93aa5c0d4d1e9806a4d6e52491c6 [id=cd89063e7d294d3fbb3b58358e781c90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.200482 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d23b5d2d67945a6ab9591969acb0d27. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:20.203385 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet ee783d2c7e274158b3de2cc186f50903 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee783d2c7e274158b3de2cc186f50903 already in progress: creating tablet
I20260618 13:59:20.206931 28800 ts_tablet_manager.cc:1929] T fc685eec02804dfabe2e25405b96def3 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.207104 28800 log.cc:1199] T fc685eec02804dfabe2e25405b96def3 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/fc685eec02804dfabe2e25405b96def3
I20260618 13:59:20.208423 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 07a4e95bd6ae47beac34fd36d6e956e3 with delete_type TABLET_DATA_DELETED (replaced by tablet eeffc96b9b97441d9301e4919d0047e1) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:20.208683 28802 tablet_replica.cc:333] T 07a4e95bd6ae47beac34fd36d6e956e3 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:20.208870 28802 raft_consensus.cc:2243] T 07a4e95bd6ae47beac34fd36d6e956e3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.209017 28802 raft_consensus.cc:2272] T 07a4e95bd6ae47beac34fd36d6e956e3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.209916 28802 ts_tablet_manager.cc:1916] T 07a4e95bd6ae47beac34fd36d6e956e3 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.210472 28800 ts_tablet_manager.cc:1950] T fc685eec02804dfabe2e25405b96def3 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:20.212733 28658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 059716cec7f94ea6ad8989fecc6f832a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.215881 28802 ts_tablet_manager.cc:1929] T 07a4e95bd6ae47beac34fd36d6e956e3 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.216018 28802 log.cc:1199] T 07a4e95bd6ae47beac34fd36d6e956e3 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/07a4e95bd6ae47beac34fd36d6e956e3
I20260618 13:59:20.216355 28802 ts_tablet_manager.cc:1950] T 07a4e95bd6ae47beac34fd36d6e956e3 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:20.217330 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 07a4e95bd6ae47beac34fd36d6e956e3 (table default.Table_f0607bf6529e4f209e00f49908420141 [id=972bc2c124ac459da06ec4093c5b564b]) successfully deleted
I20260618 13:59:20.222723 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18fb8adfc8ee40e18a1da5f978317124. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:20.227903 28216 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet eeffc96b9b97441d9301e4919d0047e1 (table default.Table_f0607bf6529e4f209e00f49908420141 [id=972bc2c124ac459da06ec4093c5b564b], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet eeffc96b9b97441d9301e4919d0047e1 consensus state has no leader
I20260618 13:59:20.232321 28800 tablet_replica.cc:333] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:20.247768 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet fc685eec02804dfabe2e25405b96def3 (table default.Table_00e4e33ddb8948a79faa0174798b215f [id=aae6979440544fe984453a915e7c2e00]) successfully deleted
I20260618 13:59:20.247753 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet 68b3bb6a4dd3446288822b5424fe5122 with delete_type TABLET_DATA_DELETED (replaced by tablet 85a3b27d914a412498919a3f65e20522) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.248159 28801 tablet_replica.cc:333] T 68b3bb6a4dd3446288822b5424fe5122 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:20.248409 28801 raft_consensus.cc:2243] T 68b3bb6a4dd3446288822b5424fe5122 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.248597 28801 raft_consensus.cc:2272] T 68b3bb6a4dd3446288822b5424fe5122 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.248740 28229 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_c1cda1e770c24c7eb9ab6c1ad8a7e786"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:20.249069 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_3bd5981b65e545459eb539217e8e3599"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:20.249408 28229 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c1cda1e770c24c7eb9ab6c1ad8a7e786 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:20.249477 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3bd5981b65e545459eb539217e8e3599 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.250679 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_f32d5142bcf64f5c9e94b015dd620e10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:20.250783 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_d52dc1e9a80f4c7db865aaf0c51bdd60"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:20.251228 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d52dc1e9a80f4c7db865aaf0c51bdd60 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:20.251230 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f32d5142bcf64f5c9e94b015dd620e10 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.252707 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_c940d6e3d5d34adfaf39c7b5b60044e1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:20.253192 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c940d6e3d5d34adfaf39c7b5b60044e1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.253269 28801 ts_tablet_manager.cc:1916] T 68b3bb6a4dd3446288822b5424fe5122 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.254338 28225 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_a6146ff7daf149b4bca36d8f09293605"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:20.254805 28225 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a6146ff7daf149b4bca36d8f09293605 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.255915 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_dea82febaef74181863a39cd4c12af41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:20.256392 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dea82febaef74181863a39cd4c12af41 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.257433 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_c18d75e5991b440caf136e9248a572da"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:20.257863 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c18d75e5991b440caf136e9248a572da in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.235625 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 411b0c333ae7450f9ada90afee553615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.259191 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_edbb2b5290e34792abb3602c8847fff8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:20.259651 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_edbb2b5290e34792abb3602c8847fff8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.259209 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_cd35c87466e64da6bd51a0ed413b9933"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:20.260986 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cd35c87466e64da6bd51a0ed413b9933 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.261054 28799 tablet_bootstrap.cc:492] T a6189fed26fb471c9724fd594054a58c P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:20.261220 28799 ts_tablet_manager.cc:1403] T a6189fed26fb471c9724fd594054a58c P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.089s user 0.020s sys 0.026s
I20260618 13:59:20.262058 28799 raft_consensus.cc:359] T a6189fed26fb471c9724fd594054a58c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.262259 28799 raft_consensus.cc:385] T a6189fed26fb471c9724fd594054a58c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.262351 28799 raft_consensus.cc:740] T a6189fed26fb471c9724fd594054a58c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.262598 28799 consensus_queue.cc:260] T a6189fed26fb471c9724fd594054a58c P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.263087 28798 tablet_bootstrap.cc:492] T fc685eec02804dfabe2e25405b96def3 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:20.263283 28798 ts_tablet_manager.cc:1403] T fc685eec02804dfabe2e25405b96def3 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.084s user 0.023s sys 0.042s
I20260618 13:59:20.264326 28798 raft_consensus.cc:359] T fc685eec02804dfabe2e25405b96def3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.264533 28798 raft_consensus.cc:385] T fc685eec02804dfabe2e25405b96def3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.264636 28798 raft_consensus.cc:740] T fc685eec02804dfabe2e25405b96def3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.264945 28798 consensus_queue.cc:260] T fc685eec02804dfabe2e25405b96def3 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.267009 28801 ts_tablet_manager.cc:1929] T 68b3bb6a4dd3446288822b5424fe5122 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.267619 28799 ts_tablet_manager.cc:1434] T a6189fed26fb471c9724fd594054a58c P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260618 13:59:20.270133 28799 tablet_bootstrap.cc:492] T fcc9e4252a1d453192e429ce4d58dfe1 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:20.272004 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet a6189fed26fb471c9724fd594054a58c with delete_type TABLET_DATA_DELETED (replaced by tablet 4ebed81a21b14db48ecb69299e0956cd) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:20.272338 28802 tablet_replica.cc:333] T a6189fed26fb471c9724fd594054a58c P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:20.272567 28802 raft_consensus.cc:2243] T a6189fed26fb471c9724fd594054a58c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.272812 28802 raft_consensus.cc:2272] T a6189fed26fb471c9724fd594054a58c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.273718 28802 ts_tablet_manager.cc:1916] T a6189fed26fb471c9724fd594054a58c P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.281777 28802 ts_tablet_manager.cc:1929] T a6189fed26fb471c9724fd594054a58c P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.281944 28802 log.cc:1199] T a6189fed26fb471c9724fd594054a58c P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/a6189fed26fb471c9724fd594054a58c
I20260618 13:59:20.282704 28800 raft_consensus.cc:2243] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.282395 28802 ts_tablet_manager.cc:1950] T a6189fed26fb471c9724fd594054a58c P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:20.284541 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet a6189fed26fb471c9724fd594054a58c (table default.Table_4ac6ae42a7f94b79abb170afc27cd978 [id=979317e20c204b109fb05c4128c0e2a3]) successfully deleted
I20260618 13:59:20.284930 28799 tablet_bootstrap.cc:654] T fcc9e4252a1d453192e429ce4d58dfe1 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.287576 28800 raft_consensus.cc:2272] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.287521 28229 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_e395544c03754cc9a2cb0514a31326e3" } new_table_name: "default.Table_a2f92d4bf5bc49aca105a2886053d230" modify_external_catalogs: true
W20260618 13:59:20.289965 28617 leader_election.cc:343] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 734374fd92d346c4a9a38c9a3d687df2
I20260618 13:59:20.290180 28617 leader_election.cc:304] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
W20260618 13:59:20.290717 28617 raft_consensus.cc:2686] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260618 13:59:20.295637 28800 ts_tablet_manager.cc:1916] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.313544 28227 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_6b0a93aa5c0d4d1e9806a4d6e52491c6" } new_table_name: "default.Table_9241cbda69c4474e8d4af9e391d5debb" modify_external_catalogs: true
I20260618 13:59:20.315192 28226 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_34aad16c9cb043f3bca2d5eda89fdecb" } new_table_name: "default.Table_9fe2f476cd334e84adf5b413afda9dcc" modify_external_catalogs: true
I20260618 13:59:20.316521 28225 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_f5ba3d11da184f25b976a8bc7c54cce1" } new_table_name: "default.Table_118ec6b705604f45a7d091a960a74735" modify_external_catalogs: true
I20260618 13:59:20.316645 28797 tablet_bootstrap.cc:654] T 68b3bb6a4dd3446288822b5424fe5122 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.317735 28222 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_31b56ffcb8004359a6089d2f1e21aba7" } new_table_name: "default.Table_ae5f307af5844ef0a03044409bb015bd" modify_external_catalogs: true
I20260618 13:59:20.320111 28223 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_4cf6e1e03c724f42a0d15aed65142720" } modify_external_catalogs: true
I20260618 13:59:20.320263 28223 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_4cf6e1e03c724f42a0d15aed65142720" } modify_external_catalogs: true
I20260618 13:59:20.267319 28798 ts_tablet_manager.cc:1434] T fc685eec02804dfabe2e25405b96def3 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 13:59:20.321771 28798 tablet_bootstrap.cc:492] T 657635f877e24b76879843d61b55dd62 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:20.323606 28798 tablet_bootstrap.cc:654] T 657635f877e24b76879843d61b55dd62 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.286083 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet fc685eec02804dfabe2e25405b96def3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.328899 28801 log.cc:1199] T 68b3bb6a4dd3446288822b5424fe5122 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/68b3bb6a4dd3446288822b5424fe5122
I20260618 13:59:20.329280 28801 ts_tablet_manager.cc:1950] T 68b3bb6a4dd3446288822b5424fe5122 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:20.330636 28220 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_e0177aee52ca4680a5786ad73020ad75" } modify_external_catalogs: true
I20260618 13:59:20.330829 28220 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_e0177aee52ca4680a5786ad73020ad75" } modify_external_catalogs: true
I20260618 13:59:20.332435 28228 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_b35e2c7c7707422d93364bf9be3bab3f" } modify_external_catalogs: true
I20260618 13:59:20.332732 28228 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_b35e2c7c7707422d93364bf9be3bab3f" } modify_external_catalogs: true
I20260618 13:59:20.333572 28801 tablet_replica.cc:333] T fc685eec02804dfabe2e25405b96def3 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:20.333762 28801 raft_consensus.cc:2243] T fc685eec02804dfabe2e25405b96def3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.333949 28801 raft_consensus.cc:2272] T fc685eec02804dfabe2e25405b96def3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.336230 28801 ts_tablet_manager.cc:1916] T fc685eec02804dfabe2e25405b96def3 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.336408 28221 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_48dff8bdaa4e430b8850f8d688195bcf" } modify_external_catalogs: true
I20260618 13:59:20.336627 28221 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_48dff8bdaa4e430b8850f8d688195bcf" } modify_external_catalogs: true
I20260618 13:59:20.337311 28224 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_bf7cbaec816a4c04a225ff31f57123ed" } modify_external_catalogs: true
I20260618 13:59:20.337497 28224 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_bf7cbaec816a4c04a225ff31f57123ed" } modify_external_catalogs: true
I20260618 13:59:20.340914 28800 ts_tablet_manager.cc:1929] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.349563 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 68b3bb6a4dd3446288822b5424fe5122 (table default.Table_94d9d315c21e4cfb9950d96262c9b6e3 [id=a874707bba8d41e7b598d7ad4a3e21f3]) successfully deleted
I20260618 13:59:20.351047 28800 log.cc:1199] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/734374fd92d346c4a9a38c9a3d687df2
I20260618 13:59:20.350023 28797 tablet_bootstrap.cc:492] T 68b3bb6a4dd3446288822b5424fe5122 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:20.351615 28797 ts_tablet_manager.cc:1403] T 68b3bb6a4dd3446288822b5424fe5122 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.287s user 0.006s sys 0.002s
I20260618 13:59:20.352376 28797 raft_consensus.cc:359] T 68b3bb6a4dd3446288822b5424fe5122 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.352564 28797 raft_consensus.cc:385] T 68b3bb6a4dd3446288822b5424fe5122 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.352682 28797 raft_consensus.cc:740] T 68b3bb6a4dd3446288822b5424fe5122 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.357007 28801 ts_tablet_manager.cc:1929] T fc685eec02804dfabe2e25405b96def3 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.357657 28800 ts_tablet_manager.cc:1950] T 734374fd92d346c4a9a38c9a3d687df2 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:20.357926 28797 consensus_queue.cc:260] T 68b3bb6a4dd3446288822b5424fe5122 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.358356 28801 log.cc:1199] T fc685eec02804dfabe2e25405b96def3 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/fc685eec02804dfabe2e25405b96def3
I20260618 13:59:20.363556 28797 ts_tablet_manager.cc:1434] T 68b3bb6a4dd3446288822b5424fe5122 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260618 13:59:20.367328 28389 tablet_service.cc:1511] Processing CreateTablet for tablet ebf502f25e9b4342bfaa58f42db7f2f5 (DEFAULT_TABLE table=default.Table_c1cda1e770c24c7eb9ab6c1ad8a7e786 [id=19c30a68cf504a179bbb2a966b68445c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.367656 28797 tablet_bootstrap.cc:492] T fcc9e4252a1d453192e429ce4d58dfe1 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:20.367820 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebf502f25e9b4342bfaa58f42db7f2f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.369405 28797 tablet_bootstrap.cc:654] T fcc9e4252a1d453192e429ce4d58dfe1 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.371748 28801 ts_tablet_manager.cc:1950] T fc685eec02804dfabe2e25405b96def3 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:20.372437 28806 raft_consensus.cc:493] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:20.372622 28806 raft_consensus.cc:515] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.373275 28806 leader_election.cc:290] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:20.374550 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "657635f877e24b76879843d61b55dd62" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:20.375406 28618 leader_election.cc:343] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:20.375926 28617 leader_election.cc:343] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 657635f877e24b76879843d61b55dd62
I20260618 13:59:20.376060 28617 leader_election.cc:304] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:20.376564 28806 raft_consensus.cc:2749] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:20.386927 28229 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 717c38fbd9cb492b9547372e2160f84f consensus state has no leader
I20260618 13:59:20.391739 28229 master_service.cc:960] ReplaceTablet: received request to replace tablet a2ca1e22c38e49d793d413e9eaaac06f from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:20.395233 28654 tablet_service.cc:1511] Processing CreateTablet for tablet ebf502f25e9b4342bfaa58f42db7f2f5 (DEFAULT_TABLE table=default.Table_c1cda1e770c24c7eb9ab6c1ad8a7e786 [id=19c30a68cf504a179bbb2a966b68445c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.395223 28529 tablet_service.cc:1511] Processing CreateTablet for tablet ebf502f25e9b4342bfaa58f42db7f2f5 (DEFAULT_TABLE table=default.Table_c1cda1e770c24c7eb9ab6c1ad8a7e786 [id=19c30a68cf504a179bbb2a966b68445c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.395828 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebf502f25e9b4342bfaa58f42db7f2f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.395843 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebf502f25e9b4342bfaa58f42db7f2f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.406898 28654 tablet_service.cc:1558] Processing DeleteTablet for tablet 68b3bb6a4dd3446288822b5424fe5122 with delete_type TABLET_DATA_DELETED (replaced by tablet 85a3b27d914a412498919a3f65e20522) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.408406 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet fc685eec02804dfabe2e25405b96def3 (table default.Table_00e4e33ddb8948a79faa0174798b215f [id=aae6979440544fe984453a915e7c2e00]) successfully deleted
I20260618 13:59:20.411934 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "397c7abe26f4451298558d6a0875b677" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:20.412292 28798 tablet_bootstrap.cc:492] T 657635f877e24b76879843d61b55dd62 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:20.412436 28798 ts_tablet_manager.cc:1403] T 657635f877e24b76879843d61b55dd62 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.091s user 0.016s sys 0.031s
W20260618 13:59:20.412971 28346 leader_election.cc:343] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:20.413151 28346 leader_election.cc:304] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:20.413450 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 102a500268574e428cb2d45e52849f57 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 102a500268574e428cb2d45e52849f57 already in progress: creating tablet
W20260618 13:59:20.413700 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 85a3b27d914a412498919a3f65e20522 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85a3b27d914a412498919a3f65e20522 already in progress: creating tablet
I20260618 13:59:20.413947 28807 raft_consensus.cc:2749] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:20.415153 28798 raft_consensus.cc:359] T 657635f877e24b76879843d61b55dd62 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.409986 28229 master_service.cc:960] ReplaceTablet: received request to replace tablet f7027a4138784d029255fc89c73104a3 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:20.415421 28798 raft_consensus.cc:385] T 657635f877e24b76879843d61b55dd62 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.415602 28798 raft_consensus.cc:740] T 657635f877e24b76879843d61b55dd62 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.415910 28798 consensus_queue.cc:260] T 657635f877e24b76879843d61b55dd62 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.416280 28800 tablet_replica.cc:333] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:20.416460 28800 raft_consensus.cc:2243] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.416635 28800 raft_consensus.cc:2272] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.420459 28798 ts_tablet_manager.cc:1434] T 657635f877e24b76879843d61b55dd62 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
W20260618 13:59:20.422001 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet a2ca1e22c38e49d793d413e9eaaac06f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a2ca1e22c38e49d793d413e9eaaac06f already in progress: creating tablet
I20260618 13:59:20.423942 28798 tablet_bootstrap.cc:492] T fcc9e4252a1d453192e429ce4d58dfe1 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:20.425771 28798 tablet_bootstrap.cc:654] T fcc9e4252a1d453192e429ce4d58dfe1 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.426959 28800 ts_tablet_manager.cc:1916] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:20.431561 28227 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 411b0c333ae7450f9ada90afee553615 consensus state has no leader
I20260618 13:59:20.437892 28227 master_service.cc:960] ReplaceTablet: received request to replace tablet b312fbf800434ecb840dd10347efe6e8 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:20.437970 28800 ts_tablet_manager.cc:1929] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.438094 28800 log.cc:1199] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/657635f877e24b76879843d61b55dd62
I20260618 13:59:20.438311 28227 master_service.cc:960] ReplaceTablet: received request to replace tablet d6e3060118e844559803a649224d7474 from {username='slave'} at 127.0.0.1:45138
W20260618 13:59:20.431773 28226 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 397c7abe26f4451298558d6a0875b677 consensus state has no leader
I20260618 13:59:20.438550 28800 ts_tablet_manager.cc:1950] T 657635f877e24b76879843d61b55dd62 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:20.438668 28226 master_service.cc:960] ReplaceTablet: received request to replace tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 from {username='slave'} at 127.0.0.1:45138
W20260618 13:59:20.422302 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet b312fbf800434ecb840dd10347efe6e8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b312fbf800434ecb840dd10347efe6e8 already in progress: creating tablet
I20260618 13:59:20.440268 28798 tablet_bootstrap.cc:492] T fcc9e4252a1d453192e429ce4d58dfe1 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:20.440428 28798 ts_tablet_manager.cc:1403] T fcc9e4252a1d453192e429ce4d58dfe1 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.000s
I20260618 13:59:20.441244 28798 raft_consensus.cc:359] T fcc9e4252a1d453192e429ce4d58dfe1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.441424 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet 657635f877e24b76879843d61b55dd62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.441466 28798 raft_consensus.cc:385] T fcc9e4252a1d453192e429ce4d58dfe1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.441640 28798 raft_consensus.cc:740] T fcc9e4252a1d453192e429ce4d58dfe1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.441783 28801 tablet_replica.cc:333] T 657635f877e24b76879843d61b55dd62 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:20.441946 28798 consensus_queue.cc:260] T fcc9e4252a1d453192e429ce4d58dfe1 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.442034 28801 raft_consensus.cc:2243] T 657635f877e24b76879843d61b55dd62 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.442239 28801 raft_consensus.cc:2272] T 657635f877e24b76879843d61b55dd62 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.443056 28801 ts_tablet_manager.cc:1916] T 657635f877e24b76879843d61b55dd62 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:20.444859 28225 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 570d36a27b7d417989ad72885d38ff63 consensus state has no leader
W20260618 13:59:20.445254 28222 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 144aa8984a44410fb3f90dc99e9b2274 consensus state has no leader
I20260618 13:59:20.449239 28801 ts_tablet_manager.cc:1929] T 657635f877e24b76879843d61b55dd62 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.449450 28801 log.cc:1199] T 657635f877e24b76879843d61b55dd62 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/657635f877e24b76879843d61b55dd62
I20260618 13:59:20.449905 28801 ts_tablet_manager.cc:1950] T 657635f877e24b76879843d61b55dd62 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:20.450866 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 37fedd27109944e2b3c870418c117758 (DEFAULT_TABLE table=default.Table_edbb2b5290e34792abb3602c8847fff8 [id=97b635f1072a4d82a82753d618600d4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.451376 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37fedd27109944e2b3c870418c117758. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.456296 28798 ts_tablet_manager.cc:1434] T fcc9e4252a1d453192e429ce4d58dfe1 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.016s user 0.001s sys 0.001s
I20260618 13:59:20.459326 28387 tablet_service.cc:1511] Processing CreateTablet for tablet d980cda1c60e4edca7b4d7537f4f5837 (DEFAULT_TABLE table=default.Table_3bd5981b65e545459eb539217e8e3599 [id=7c39b3c432e944bca05f7b2070a79ff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.459836 28387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d980cda1c60e4edca7b4d7537f4f5837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.461333 28798 tablet_bootstrap.cc:492] T a6189fed26fb471c9724fd594054a58c P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:20.462990 28798 tablet_bootstrap.cc:654] T a6189fed26fb471c9724fd594054a58c P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.463157 28386 tablet_service.cc:1511] Processing CreateTablet for tablet ffd7d5ebaa94416a8eaf3e61bed945e1 (DEFAULT_TABLE table=default.Table_c18d75e5991b440caf136e9248a572da [id=5a3cb5befb614a70b4ecf8ccfedf8ecd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.463511 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffd7d5ebaa94416a8eaf3e61bed945e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.463963 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet fcc9e4252a1d453192e429ce4d58dfe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.464162 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 37fedd27109944e2b3c870418c117758 (DEFAULT_TABLE table=default.Table_edbb2b5290e34792abb3602c8847fff8 [id=97b635f1072a4d82a82753d618600d4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.464260 28801 tablet_replica.cc:333] T fcc9e4252a1d453192e429ce4d58dfe1 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:20.464409 28801 raft_consensus.cc:2243] T fcc9e4252a1d453192e429ce4d58dfe1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.464499 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37fedd27109944e2b3c870418c117758. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.464608 28801 raft_consensus.cc:2272] T fcc9e4252a1d453192e429ce4d58dfe1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.465304 28801 ts_tablet_manager.cc:1916] T fcc9e4252a1d453192e429ce4d58dfe1 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.467063 28800 tablet_replica.cc:333] T 68b3bb6a4dd3446288822b5424fe5122 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:20.467257 28800 raft_consensus.cc:2243] T 68b3bb6a4dd3446288822b5424fe5122 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.467432 28800 raft_consensus.cc:2272] T 68b3bb6a4dd3446288822b5424fe5122 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.471364 28654 tablet_service.cc:1511] Processing CreateTablet for tablet 37fedd27109944e2b3c870418c117758 (DEFAULT_TABLE table=default.Table_edbb2b5290e34792abb3602c8847fff8 [id=97b635f1072a4d82a82753d618600d4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.471822 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37fedd27109944e2b3c870418c117758. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.473692 28801 ts_tablet_manager.cc:1929] T fcc9e4252a1d453192e429ce4d58dfe1 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.471474 28800 ts_tablet_manager.cc:1916] T 68b3bb6a4dd3446288822b5424fe5122 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.479772 28800 ts_tablet_manager.cc:1929] T 68b3bb6a4dd3446288822b5424fe5122 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.481666 28801 log.cc:1199] T fcc9e4252a1d453192e429ce4d58dfe1 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/fcc9e4252a1d453192e429ce4d58dfe1
I20260618 13:59:20.482343 28801 ts_tablet_manager.cc:1950] T fcc9e4252a1d453192e429ce4d58dfe1 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:20.483114 28800 log.cc:1199] T 68b3bb6a4dd3446288822b5424fe5122 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/68b3bb6a4dd3446288822b5424fe5122
I20260618 13:59:20.483634 28800 ts_tablet_manager.cc:1950] T 68b3bb6a4dd3446288822b5424fe5122 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:20.483777 28799 tablet_bootstrap.cc:492] T fcc9e4252a1d453192e429ce4d58dfe1 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:20.484004 28799 ts_tablet_manager.cc:1403] T fcc9e4252a1d453192e429ce4d58dfe1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.214s user 0.055s sys 0.131s
I20260618 13:59:20.484839 28799 raft_consensus.cc:359] T fcc9e4252a1d453192e429ce4d58dfe1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.485390 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 657635f877e24b76879843d61b55dd62 (table default.Table_7a5b9ad4d9b44f358878aae89187c35a [id=74c1ef2bdbe342ff85c1d825b800a1cf]) successfully deleted
I20260618 13:59:20.485847 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet fcc9e4252a1d453192e429ce4d58dfe1 (table default.Table_0700cfccef1048aea83be790024db03d [id=018abd3973f146e9880ac9ba37ddfd20]) successfully deleted
I20260618 13:59:20.486630 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 657635f877e24b76879843d61b55dd62 (table default.Table_7a5b9ad4d9b44f358878aae89187c35a [id=74c1ef2bdbe342ff85c1d825b800a1cf]) successfully deleted
W20260618 13:59:20.486876 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet ee783d2c7e274158b3de2cc186f50903 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee783d2c7e274158b3de2cc186f50903 already in progress: creating tablet
I20260618 13:59:20.486828 28655 tablet_service.cc:1511] Processing CreateTablet for tablet d980cda1c60e4edca7b4d7537f4f5837 (DEFAULT_TABLE table=default.Table_3bd5981b65e545459eb539217e8e3599 [id=7c39b3c432e944bca05f7b2070a79ff9]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:20.487084 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet a6189fed26fb471c9724fd594054a58c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a6189fed26fb471c9724fd594054a58c already in progress: creating tablet
I20260618 13:59:20.487223 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d980cda1c60e4edca7b4d7537f4f5837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.487353 28799 raft_consensus.cc:385] T fcc9e4252a1d453192e429ce4d58dfe1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.487422 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 68b3bb6a4dd3446288822b5424fe5122 (table default.Table_94d9d315c21e4cfb9950d96262c9b6e3 [id=a874707bba8d41e7b598d7ad4a3e21f3]) successfully deleted
I20260618 13:59:20.487454 28799 raft_consensus.cc:740] T fcc9e4252a1d453192e429ce4d58dfe1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.487699 28799 consensus_queue.cc:260] T fcc9e4252a1d453192e429ce4d58dfe1 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.488202 28799 ts_tablet_manager.cc:1434] T fcc9e4252a1d453192e429ce4d58dfe1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 13:59:20.489117 28799 tablet_bootstrap.cc:492] T f7027a4138784d029255fc89c73104a3 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:20.489406 28386 tablet_service.cc:1511] Processing CreateTablet for tablet 1f0109536c24485c9934193f2ef8aeae (DEFAULT_TABLE table=default.Table_d52dc1e9a80f4c7db865aaf0c51bdd60 [id=cfb4ca8c4e884ccc8602be52d0b39100]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.489781 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f0109536c24485c9934193f2ef8aeae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.490886 28799 tablet_bootstrap.cc:654] T f7027a4138784d029255fc89c73104a3 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.490873 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_ceb33d60d86e4fd898ae67c78da94a55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:20.491421 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ceb33d60d86e4fd898ae67c78da94a55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.496395 28387 tablet_service.cc:1558] Processing DeleteTablet for tablet fcc9e4252a1d453192e429ce4d58dfe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:20.496733 28802 tablet_replica.cc:333] T fcc9e4252a1d453192e429ce4d58dfe1 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:20.496933 28802 raft_consensus.cc:2243] T fcc9e4252a1d453192e429ce4d58dfe1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.496959 28526 tablet_service.cc:1511] Processing CreateTablet for tablet d980cda1c60e4edca7b4d7537f4f5837 (DEFAULT_TABLE table=default.Table_3bd5981b65e545459eb539217e8e3599 [id=7c39b3c432e944bca05f7b2070a79ff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.497113 28802 raft_consensus.cc:2272] T fcc9e4252a1d453192e429ce4d58dfe1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.497406 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d980cda1c60e4edca7b4d7537f4f5837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.499310 28799 tablet_bootstrap.cc:492] T f7027a4138784d029255fc89c73104a3 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:20.499511 28799 ts_tablet_manager.cc:1403] T f7027a4138784d029255fc89c73104a3 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.001s
I20260618 13:59:20.500315 28799 raft_consensus.cc:359] T f7027a4138784d029255fc89c73104a3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.500564 28799 raft_consensus.cc:385] T f7027a4138784d029255fc89c73104a3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.500733 28799 raft_consensus.cc:740] T f7027a4138784d029255fc89c73104a3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.501060 28799 consensus_queue.cc:260] T f7027a4138784d029255fc89c73104a3 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.501137 28802 ts_tablet_manager.cc:1916] T fcc9e4252a1d453192e429ce4d58dfe1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.502990 28799 ts_tablet_manager.cc:1434] T f7027a4138784d029255fc89c73104a3 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:20.503294 28799 tablet_bootstrap.cc:492] T f4447a3e87bd476e84f59dd4c4185b26 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:20.505018 28799 tablet_bootstrap.cc:654] T f4447a3e87bd476e84f59dd4c4185b26 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.516300 28802 ts_tablet_manager.cc:1929] T fcc9e4252a1d453192e429ce4d58dfe1 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.516644 28802 log.cc:1199] T fcc9e4252a1d453192e429ce4d58dfe1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/fcc9e4252a1d453192e429ce4d58dfe1
I20260618 13:59:20.517205 28802 ts_tablet_manager.cc:1950] T fcc9e4252a1d453192e429ce4d58dfe1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:20.518515 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet fcc9e4252a1d453192e429ce4d58dfe1 (table default.Table_0700cfccef1048aea83be790024db03d [id=018abd3973f146e9880ac9ba37ddfd20]) successfully deleted
I20260618 13:59:20.519304 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet f7027a4138784d029255fc89c73104a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:20.520704 28797 tablet_bootstrap.cc:492] T fcc9e4252a1d453192e429ce4d58dfe1 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:20.520932 28797 ts_tablet_manager.cc:1403] T fcc9e4252a1d453192e429ce4d58dfe1 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.153s user 0.060s sys 0.056s
I20260618 13:59:20.521615 28389 tablet_service.cc:1511] Processing CreateTablet for tablet b1f734a3f29b42e591d5564e99292b5e (DEFAULT_TABLE table=default.Table_c940d6e3d5d34adfaf39c7b5b60044e1 [id=803d5220ce1a4dcfaf27d93c574be2ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.521857 28797 raft_consensus.cc:359] T fcc9e4252a1d453192e429ce4d58dfe1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.522058 28797 raft_consensus.cc:385] T fcc9e4252a1d453192e429ce4d58dfe1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.522145 28797 raft_consensus.cc:740] T fcc9e4252a1d453192e429ce4d58dfe1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.522253 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1f734a3f29b42e591d5564e99292b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.522398 28797 consensus_queue.cc:260] T fcc9e4252a1d453192e429ce4d58dfe1 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.522898 28797 ts_tablet_manager.cc:1434] T fcc9e4252a1d453192e429ce4d58dfe1 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 13:59:20.523211 28797 tablet_bootstrap.cc:492] T a6189fed26fb471c9724fd594054a58c P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:20.524933 28797 tablet_bootstrap.cc:654] T a6189fed26fb471c9724fd594054a58c P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.531134 28526 tablet_service.cc:1511] Processing CreateTablet for tablet ffd7d5ebaa94416a8eaf3e61bed945e1 (DEFAULT_TABLE table=default.Table_c18d75e5991b440caf136e9248a572da [id=5a3cb5befb614a70b4ecf8ccfedf8ecd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.531286 28655 tablet_service.cc:1511] Processing CreateTablet for tablet ffd7d5ebaa94416a8eaf3e61bed945e1 (DEFAULT_TABLE table=default.Table_c18d75e5991b440caf136e9248a572da [id=5a3cb5befb614a70b4ecf8ccfedf8ecd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.531591 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffd7d5ebaa94416a8eaf3e61bed945e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.531694 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffd7d5ebaa94416a8eaf3e61bed945e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.541249 28802 tablet_replica.cc:333] T f7027a4138784d029255fc89c73104a3 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:20.541600 28802 raft_consensus.cc:2243] T f7027a4138784d029255fc89c73104a3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.541855 28802 raft_consensus.cc:2272] T f7027a4138784d029255fc89c73104a3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.542812 28802 ts_tablet_manager.cc:1916] T f7027a4138784d029255fc89c73104a3 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.543628 28661 tablet_service.cc:1558] Processing DeleteTablet for tablet fcc9e4252a1d453192e429ce4d58dfe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.543759 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 1f0109536c24485c9934193f2ef8aeae (DEFAULT_TABLE table=default.Table_d52dc1e9a80f4c7db865aaf0c51bdd60 [id=cfb4ca8c4e884ccc8602be52d0b39100]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.543951 28800 tablet_replica.cc:333] T fcc9e4252a1d453192e429ce4d58dfe1 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:20.544149 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f0109536c24485c9934193f2ef8aeae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.544174 28800 raft_consensus.cc:2243] T fcc9e4252a1d453192e429ce4d58dfe1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.544416 28800 raft_consensus.cc:2272] T fcc9e4252a1d453192e429ce4d58dfe1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.545135 28800 ts_tablet_manager.cc:1916] T fcc9e4252a1d453192e429ce4d58dfe1 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.545887 28229 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_2bf9616ba425458d8c7a4e9a6854d122"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:20.545902 28225 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_067483ebd76c49e595543cc25ff1db56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:20.546653 28225 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_067483ebd76c49e595543cc25ff1db56 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.546948 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_8f22977cfa824da99508b4063a05f8e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:20.547539 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f22977cfa824da99508b4063a05f8e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.547765 28655 tablet_service.cc:1511] Processing CreateTablet for tablet 1f0109536c24485c9934193f2ef8aeae (DEFAULT_TABLE table=default.Table_d52dc1e9a80f4c7db865aaf0c51bdd60 [id=cfb4ca8c4e884ccc8602be52d0b39100]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:20.546653 28229 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2bf9616ba425458d8c7a4e9a6854d122 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.548249 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f0109536c24485c9934193f2ef8aeae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.551784 28800 ts_tablet_manager.cc:1929] T fcc9e4252a1d453192e429ce4d58dfe1 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.551908 28800 log.cc:1199] T fcc9e4252a1d453192e429ce4d58dfe1 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/fcc9e4252a1d453192e429ce4d58dfe1
I20260618 13:59:20.552309 28800 ts_tablet_manager.cc:1950] T fcc9e4252a1d453192e429ce4d58dfe1 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:20.552650 28220 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 059716cec7f94ea6ad8989fecc6f832a
I20260618 13:59:20.553340 28386 tablet_service.cc:1511] Processing CreateTablet for tablet e1a75029992645c5b942c0ec07862bb4 (DEFAULT_TABLE table=default.Table_dea82febaef74181863a39cd4c12af41 [id=bd90cbbd5592449480ef377189a9c2de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.553586 28387 tablet_service.cc:1558] Processing DeleteTablet for tablet 059716cec7f94ea6ad8989fecc6f832a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:20.553802 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a75029992645c5b942c0ec07862bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.555536 28387 tablet_service.cc:1511] Processing CreateTablet for tablet bd6b0cdffd5a44b5b45b885f6e88eb9d (DEFAULT_TABLE table=default.Table_f32d5142bcf64f5c9e94b015dd620e10 [id=cfb012a9137e481eb0e03145a6404788]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.555914 28387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd6b0cdffd5a44b5b45b885f6e88eb9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.556860 28802 ts_tablet_manager.cc:1929] T f7027a4138784d029255fc89c73104a3 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.557024 28802 log.cc:1199] T f7027a4138784d029255fc89c73104a3 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/f7027a4138784d029255fc89c73104a3
I20260618 13:59:20.557561 28802 ts_tablet_manager.cc:1950] T f7027a4138784d029255fc89c73104a3 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:20.567770 28798 tablet_bootstrap.cc:492] T a6189fed26fb471c9724fd594054a58c P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:20.567988 28798 ts_tablet_manager.cc:1403] T a6189fed26fb471c9724fd594054a58c P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.107s user 0.016s sys 0.028s
I20260618 13:59:20.569146 28798 raft_consensus.cc:359] T a6189fed26fb471c9724fd594054a58c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.569398 28798 raft_consensus.cc:385] T a6189fed26fb471c9724fd594054a58c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.569520 28798 raft_consensus.cc:740] T a6189fed26fb471c9724fd594054a58c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.569876 28798 consensus_queue.cc:260] T a6189fed26fb471c9724fd594054a58c P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.570221 28224 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 18fb8adfc8ee40e18a1da5f978317124
I20260618 13:59:20.570997 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_03e5ab6f90514a49815d0f0c6c8e5f80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:20.571596 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03e5ab6f90514a49815d0f0c6c8e5f80 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.572741 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_d81b926f20864cfe9915d642c53fa6f2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:20.572921 28654 tablet_service.cc:1511] Processing CreateTablet for tablet b1f734a3f29b42e591d5564e99292b5e (DEFAULT_TABLE table=default.Table_c940d6e3d5d34adfaf39c7b5b60044e1 [id=803d5220ce1a4dcfaf27d93c574be2ac]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:20.573216 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d81b926f20864cfe9915d642c53fa6f2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.573307 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1f734a3f29b42e591d5564e99292b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.575500 28526 tablet_service.cc:1511] Processing CreateTablet for tablet b1f734a3f29b42e591d5564e99292b5e (DEFAULT_TABLE table=default.Table_c940d6e3d5d34adfaf39c7b5b60044e1 [id=803d5220ce1a4dcfaf27d93c574be2ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.575865 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1f734a3f29b42e591d5564e99292b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.578887 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet 059716cec7f94ea6ad8989fecc6f832a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.578891 28661 tablet_service.cc:1558] Processing DeleteTablet for tablet 059716cec7f94ea6ad8989fecc6f832a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.579948 28529 tablet_service.cc:1511] Processing CreateTablet for tablet e1a75029992645c5b942c0ec07862bb4 (DEFAULT_TABLE table=default.Table_dea82febaef74181863a39cd4c12af41 [id=bd90cbbd5592449480ef377189a9c2de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.580176 28661 tablet_service.cc:1511] Processing CreateTablet for tablet e1a75029992645c5b942c0ec07862bb4 (DEFAULT_TABLE table=default.Table_dea82febaef74181863a39cd4c12af41 [id=bd90cbbd5592449480ef377189a9c2de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.580273 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a75029992645c5b942c0ec07862bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.580606 28661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a75029992645c5b942c0ec07862bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.580992 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet f7027a4138784d029255fc89c73104a3 (table default.Table_7559b439752d4fa68c4fb6afee136bfc [id=61b2edf42a374060934bf910c3f2dde5]) successfully deleted
W20260618 13:59:20.581255 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 059716cec7f94ea6ad8989fecc6f832a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 059716cec7f94ea6ad8989fecc6f832a already in progress: creating tablet
I20260618 13:59:20.582561 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet 18fb8adfc8ee40e18a1da5f978317124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:20.583154 28387 tablet_service.cc:1511] Processing CreateTablet for tablet 668c6c6f2034484ca5ccdd51c22dea1d (DEFAULT_TABLE table=default.Table_a6146ff7daf149b4bca36d8f09293605 [id=aa7194d6b3b2419294e5aa57d0043a11]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:20.583842 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 18fb8adfc8ee40e18a1da5f978317124 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 18fb8adfc8ee40e18a1da5f978317124 already in progress: creating tablet
I20260618 13:59:20.584314 28387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 668c6c6f2034484ca5ccdd51c22dea1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.586364 28798 ts_tablet_manager.cc:1434] T a6189fed26fb471c9724fd594054a58c P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.018s user 0.001s sys 0.002s
I20260618 13:59:20.586735 28391 tablet_service.cc:1511] Processing CreateTablet for tablet b739fa1847184290833e13a2515f3a3a (DEFAULT_TABLE table=default.Table_cd35c87466e64da6bd51a0ed413b9933 [id=4961f93e6ad64ec7b3bddcc62db96522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.587050 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b739fa1847184290833e13a2515f3a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.589411 28798 tablet_bootstrap.cc:492] T f7027a4138784d029255fc89c73104a3 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
W20260618 13:59:20.590042 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 059716cec7f94ea6ad8989fecc6f832a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 059716cec7f94ea6ad8989fecc6f832a already in progress: creating tablet
I20260618 13:59:20.590142 28529 tablet_service.cc:1511] Processing CreateTablet for tablet bd6b0cdffd5a44b5b45b885f6e88eb9d (DEFAULT_TABLE table=default.Table_f32d5142bcf64f5c9e94b015dd620e10 [id=cfb012a9137e481eb0e03145a6404788]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.590379 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet fcc9e4252a1d453192e429ce4d58dfe1 (table default.Table_0700cfccef1048aea83be790024db03d [id=018abd3973f146e9880ac9ba37ddfd20]) successfully deleted
I20260618 13:59:20.590744 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd6b0cdffd5a44b5b45b885f6e88eb9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.594887 28661 tablet_service.cc:1511] Processing CreateTablet for tablet bd6b0cdffd5a44b5b45b885f6e88eb9d (DEFAULT_TABLE table=default.Table_f32d5142bcf64f5c9e94b015dd620e10 [id=cfb012a9137e481eb0e03145a6404788]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.595258 28661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd6b0cdffd5a44b5b45b885f6e88eb9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.597303 28048 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu
/tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.100.61:45967
--webserver_interface=127.27.100.61
--webserver_port=35663
--builtin_ntp_servers=127.27.100.20:36797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.100.62:43001,127.27.100.61:45967,127.27.100.60:42083
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260618 13:59:20.597379 28798 tablet_bootstrap.cc:654] T f7027a4138784d029255fc89c73104a3 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.599910 28799 tablet_bootstrap.cc:492] T f4447a3e87bd476e84f59dd4c4185b26 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:20.600090 28799 ts_tablet_manager.cc:1403] T f4447a3e87bd476e84f59dd4c4185b26 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.097s user 0.017s sys 0.029s
I20260618 13:59:20.600953 28799 raft_consensus.cc:359] T f4447a3e87bd476e84f59dd4c4185b26 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.601179 28799 raft_consensus.cc:385] T f4447a3e87bd476e84f59dd4c4185b26 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.601251 28799 raft_consensus.cc:740] T f4447a3e87bd476e84f59dd4c4185b26 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.601549 28799 consensus_queue.cc:260] T f4447a3e87bd476e84f59dd4c4185b26 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.602111 28799 ts_tablet_manager.cc:1434] T f4447a3e87bd476e84f59dd4c4185b26 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:20.602475 28799 tablet_bootstrap.cc:492] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:20.602963 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 059716cec7f94ea6ad8989fecc6f832a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 059716cec7f94ea6ad8989fecc6f832a already in progress: creating tablet
I20260618 13:59:20.605168 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet 18fb8adfc8ee40e18a1da5f978317124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.605657 28650 tablet_service.cc:1511] Processing CreateTablet for tablet 668c6c6f2034484ca5ccdd51c22dea1d (DEFAULT_TABLE table=default.Table_a6146ff7daf149b4bca36d8f09293605 [id=aa7194d6b3b2419294e5aa57d0043a11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.606082 28650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 668c6c6f2034484ca5ccdd51c22dea1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.606765 28656 tablet_service.cc:1511] Processing CreateTablet for tablet b739fa1847184290833e13a2515f3a3a (DEFAULT_TABLE table=default.Table_cd35c87466e64da6bd51a0ed413b9933 [id=4961f93e6ad64ec7b3bddcc62db96522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.607100 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b739fa1847184290833e13a2515f3a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.609293 28221 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 1d23b5d2d67945a6ab9591969acb0d27
I20260618 13:59:20.611455 28228 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet d6e3060118e844559803a649224d7474
I20260618 13:59:20.611609 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 668c6c6f2034484ca5ccdd51c22dea1d (DEFAULT_TABLE table=default.Table_a6146ff7daf149b4bca36d8f09293605 [id=aa7194d6b3b2419294e5aa57d0043a11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.611985 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 668c6c6f2034484ca5ccdd51c22dea1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.612094 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_4ed45da8b1ed4ff4ad05e46947caf9df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:20.612612 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4ed45da8b1ed4ff4ad05e46947caf9df in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.613489 28799 tablet_bootstrap.cc:654] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.614110 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_30c1b9baedbf4ed588651ac692eff7b6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:20.614603 28525 tablet_service.cc:1558] Processing DeleteTablet for tablet 18fb8adfc8ee40e18a1da5f978317124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.614732 28522 tablet_service.cc:1511] Processing CreateTablet for tablet b739fa1847184290833e13a2515f3a3a (DEFAULT_TABLE table=default.Table_cd35c87466e64da6bd51a0ed413b9933 [id=4961f93e6ad64ec7b3bddcc62db96522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.615069 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b739fa1847184290833e13a2515f3a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.615162 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_0a13b90182654346acbbc8f47a56436d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:20.615567 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a13b90182654346acbbc8f47a56436d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.616689 28797 tablet_bootstrap.cc:492] T a6189fed26fb471c9724fd594054a58c P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:20.616801 28797 ts_tablet_manager.cc:1403] T a6189fed26fb471c9724fd594054a58c P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.094s user 0.022s sys 0.008s
I20260618 13:59:20.617513 28797 raft_consensus.cc:359] T a6189fed26fb471c9724fd594054a58c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.617691 28797 raft_consensus.cc:385] T a6189fed26fb471c9724fd594054a58c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.617777 28797 raft_consensus.cc:740] T a6189fed26fb471c9724fd594054a58c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.618000 28797 consensus_queue.cc:260] T a6189fed26fb471c9724fd594054a58c P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.618551 28797 ts_tablet_manager.cc:1434] T a6189fed26fb471c9724fd594054a58c P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260618 13:59:20.618711 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_30c1b9baedbf4ed588651ac692eff7b6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.623595 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet d6e3060118e844559803a649224d7474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:20.626103 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet d6e3060118e844559803a649224d7474 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d6e3060118e844559803a649224d7474 already in progress: creating tablet
I20260618 13:59:20.626575 28799 tablet_bootstrap.cc:492] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:20.626732 28799 ts_tablet_manager.cc:1403] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.008s
I20260618 13:59:20.627521 28799 raft_consensus.cc:359] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.627686 28799 raft_consensus.cc:385] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.627774 28799 raft_consensus.cc:740] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.627767 28223 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet f4447a3e87bd476e84f59dd4c4185b26
I20260618 13:59:20.627974 28799 consensus_queue.cc:260] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.628458 28799 ts_tablet_manager.cc:1434] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:20.628734 28799 tablet_bootstrap.cc:492] T 91c250dfbd72416d9d74f6cbb67b2bdf P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:20.629374 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_1de2d9113caa4693a242a5ec6f6ee0c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:20.629863 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1de2d9113caa4693a242a5ec6f6ee0c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.630407 28799 tablet_bootstrap.cc:654] T 91c250dfbd72416d9d74f6cbb67b2bdf P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.630991 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet f4447a3e87bd476e84f59dd4c4185b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:20.631450 28802 tablet_replica.cc:333] T f4447a3e87bd476e84f59dd4c4185b26 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:20.631633 28802 raft_consensus.cc:2243] T f4447a3e87bd476e84f59dd4c4185b26 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.631783 28802 raft_consensus.cc:2272] T f4447a3e87bd476e84f59dd4c4185b26 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.632536 28802 ts_tablet_manager.cc:1916] T f4447a3e87bd476e84f59dd4c4185b26 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.641170 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d23b5d2d67945a6ab9591969acb0d27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:20.642912 28797 tablet_bootstrap.cc:492] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:20.645320 28797 tablet_bootstrap.cc:654] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.646337 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:20.651057 28802 ts_tablet_manager.cc:1929] T f4447a3e87bd476e84f59dd4c4185b26 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.651265 28802 log.cc:1199] T f4447a3e87bd476e84f59dd4c4185b26 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/f4447a3e87bd476e84f59dd4c4185b26
I20260618 13:59:20.651752 28802 ts_tablet_manager.cc:1950] T f4447a3e87bd476e84f59dd4c4185b26 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:20.652792 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet f4447a3e87bd476e84f59dd4c4185b26 (table default.Table_4cf6e1e03c724f42a0d15aed65142720 [id=80246ec4fbca44248528c2c379034cee]) successfully deleted
W20260618 13:59:20.653434 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 1d23b5d2d67945a6ab9591969acb0d27 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d23b5d2d67945a6ab9591969acb0d27 already in progress: creating tablet
I20260618 13:59:20.653811 28802 tablet_replica.cc:333] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:20.654034 28802 raft_consensus.cc:2243] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.654311 28802 raft_consensus.cc:2272] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.655259 28802 ts_tablet_manager.cc:1916] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.661530 28797 tablet_bootstrap.cc:492] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:20.661785 28797 ts_tablet_manager.cc:1403] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.010s
I20260618 13:59:20.662606 28797 raft_consensus.cc:359] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.663341 28797 raft_consensus.cc:385] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.664100 28799 tablet_bootstrap.cc:492] T 91c250dfbd72416d9d74f6cbb67b2bdf P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:20.664255 28797 raft_consensus.cc:740] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.664711 28797 consensus_queue.cc:260] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.664455 28802 ts_tablet_manager.cc:1929] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.665165 28799 ts_tablet_manager.cc:1403] T 91c250dfbd72416d9d74f6cbb67b2bdf P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.036s user 0.005s sys 0.011s
I20260618 13:59:20.665184 28802 log.cc:1199] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/9fd3fbcf0d9745a1b3d3408f95022bf6
I20260618 13:59:20.665732 28802 ts_tablet_manager.cc:1950] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:20.665921 28806 raft_consensus.cc:493] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:20.666122 28806 raft_consensus.cc:515] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.667032 28806 leader_election.cc:290] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:20.667145 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 (table default.Table_e10ecf3a190148d0b17f97204f155353 [id=179292b8d0574f7bb5b6edb5dcad5cb9]) successfully deleted
I20260618 13:59:20.667570 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7027a4138784d029255fc89c73104a3" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:20.667888 28797 ts_tablet_manager.cc:1434] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260618 13:59:20.668216 28797 tablet_bootstrap.cc:492] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:20.668912 28798 tablet_bootstrap.cc:492] T f7027a4138784d029255fc89c73104a3 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:20.669569 28798 ts_tablet_manager.cc:1403] T f7027a4138784d029255fc89c73104a3 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.080s user 0.004s sys 0.004s
I20260618 13:59:20.669756 28797 tablet_bootstrap.cc:654] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.670748 28798 raft_consensus.cc:359] T f7027a4138784d029255fc89c73104a3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.671010 28799 raft_consensus.cc:359] T 91c250dfbd72416d9d74f6cbb67b2bdf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.671203 28799 raft_consensus.cc:385] T 91c250dfbd72416d9d74f6cbb67b2bdf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.671277 28799 raft_consensus.cc:740] T 91c250dfbd72416d9d74f6cbb67b2bdf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.671531 28799 consensus_queue.cc:260] T 91c250dfbd72416d9d74f6cbb67b2bdf P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:20.667898 28617 leader_election.cc:343] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: f7027a4138784d029255fc89c73104a3
W20260618 13:59:20.671841 28618 leader_election.cc:343] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:20.672022 28799 ts_tablet_manager.cc:1434] T 91c250dfbd72416d9d74f6cbb67b2bdf P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260618 13:59:20.672312 28799 tablet_bootstrap.cc:492] T 587b82a6da8d4fe48b26070803831d9d P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:20.672282 28618 leader_election.cc:304] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:20.672747 28806 raft_consensus.cc:2749] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:20.673236 28798 raft_consensus.cc:385] T f7027a4138784d029255fc89c73104a3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.673508 28798 raft_consensus.cc:740] T f7027a4138784d029255fc89c73104a3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.673887 28799 tablet_bootstrap.cc:654] T 587b82a6da8d4fe48b26070803831d9d P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.674100 28798 consensus_queue.cc:260] T f7027a4138784d029255fc89c73104a3 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.674885 28798 ts_tablet_manager.cc:1434] T f7027a4138784d029255fc89c73104a3 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260618 13:59:20.677567 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d23b5d2d67945a6ab9591969acb0d27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.675441 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d23b5d2d67945a6ab9591969acb0d27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.681208 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet a6189fed26fb471c9724fd594054a58c with delete_type TABLET_DATA_DELETED (replaced by tablet 4ebed81a21b14db48ecb69299e0956cd) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.681663 28798 tablet_bootstrap.cc:492] T 59613383d2294f9ab6bce6321aad44bb P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:20.682571 28801 tablet_replica.cc:333] T a6189fed26fb471c9724fd594054a58c P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:20.682780 28801 raft_consensus.cc:2243] T a6189fed26fb471c9724fd594054a58c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.682933 28801 raft_consensus.cc:2272] T a6189fed26fb471c9724fd594054a58c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.683380 28798 tablet_bootstrap.cc:654] T 59613383d2294f9ab6bce6321aad44bb P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.683820 28801 ts_tablet_manager.cc:1916] T a6189fed26fb471c9724fd594054a58c P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.684996 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet d6e3060118e844559803a649224d7474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.685732 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 91c250dfbd72416d9d74f6cbb67b2bdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:20.686172 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet f4447a3e87bd476e84f59dd4c4185b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:20.687227 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 18fb8adfc8ee40e18a1da5f978317124 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 18fb8adfc8ee40e18a1da5f978317124 already in progress: creating tablet
W20260618 13:59:20.687815 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 1d23b5d2d67945a6ab9591969acb0d27 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d23b5d2d67945a6ab9591969acb0d27 already in progress: creating tablet
W20260618 13:59:20.688223 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 18fb8adfc8ee40e18a1da5f978317124 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 18fb8adfc8ee40e18a1da5f978317124 already in progress: creating tablet
W20260618 13:59:20.688663 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 1d23b5d2d67945a6ab9591969acb0d27 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d23b5d2d67945a6ab9591969acb0d27 already in progress: creating tablet
I20260618 13:59:20.689144 28221 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_cd191bffa0274a44be88412fadf7faa3" } new_table_name: "default.Table_5b9198f79dca44a9a66c38929d637bde" modify_external_catalogs: true
I20260618 13:59:20.689448 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet f4447a3e87bd476e84f59dd4c4185b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.689932 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet d6e3060118e844559803a649224d7474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.691473 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet f7027a4138784d029255fc89c73104a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.691632 28802 tablet_replica.cc:333] T 91c250dfbd72416d9d74f6cbb67b2bdf P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:20.691823 28802 raft_consensus.cc:2243] T 91c250dfbd72416d9d74f6cbb67b2bdf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.691996 28802 raft_consensus.cc:2272] T 91c250dfbd72416d9d74f6cbb67b2bdf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.692593 28801 ts_tablet_manager.cc:1929] T a6189fed26fb471c9724fd594054a58c P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.693574 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet f7027a4138784d029255fc89c73104a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.693818 28801 log.cc:1199] T a6189fed26fb471c9724fd594054a58c P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/a6189fed26fb471c9724fd594054a58c
I20260618 13:59:20.693822 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet a6189fed26fb471c9724fd594054a58c with delete_type TABLET_DATA_DELETED (replaced by tablet 4ebed81a21b14db48ecb69299e0956cd) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.694031 28800 tablet_replica.cc:333] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:20.694351 28801 ts_tablet_manager.cc:1950] T a6189fed26fb471c9724fd594054a58c P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:20.697502 28802 ts_tablet_manager.cc:1916] T 91c250dfbd72416d9d74f6cbb67b2bdf P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.697813 28801 tablet_replica.cc:333] T f7027a4138784d029255fc89c73104a3 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:20.698061 28801 raft_consensus.cc:2243] T f7027a4138784d029255fc89c73104a3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.699112 28225 master_service.cc:960] ReplaceTablet: received request to replace tablet 07a4e95bd6ae47beac34fd36d6e956e3 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:20.699376 28801 raft_consensus.cc:2272] T f7027a4138784d029255fc89c73104a3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.699626 28225 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_9fe2f476cd334e84adf5b413afda9dcc" } new_table_name: "default.Table_2be2c24d66ae4f1d8bf1e78d134841c7" modify_external_catalogs: true
I20260618 13:59:20.700484 28801 ts_tablet_manager.cc:1916] T f7027a4138784d029255fc89c73104a3 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.701013 28226 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_479c1c1ed7d14546932db3cd2001df61" } new_table_name: "default.Table_dd833764ce1041cba71575be46d771a9" modify_external_catalogs: true
I20260618 13:59:20.702310 28229 master_service.cc:960] ReplaceTablet: received request to replace tablet 8c9d89fa75174a969ded07e0596db99b from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:20.702800 28229 master_service.cc:960] ReplaceTablet: received request to replace tablet d5cb3137659a4481840fe04174ef682d from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:20.703330 28224 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_ae5f307af5844ef0a03044409bb015bd" } new_table_name: "default.Table_56009fb08789451f82400e86054dd066" modify_external_catalogs: true
I20260618 13:59:20.704293 28802 ts_tablet_manager.cc:1929] T 91c250dfbd72416d9d74f6cbb67b2bdf P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.704432 28802 log.cc:1199] T 91c250dfbd72416d9d74f6cbb67b2bdf P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/91c250dfbd72416d9d74f6cbb67b2bdf
I20260618 13:59:20.704527 28220 master_service.cc:960] ReplaceTablet: received request to replace tablet b1f734a3f29b42e591d5564e99292b5e from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:20.704847 28802 ts_tablet_manager.cc:1950] T 91c250dfbd72416d9d74f6cbb67b2bdf P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:20.705801 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 91c250dfbd72416d9d74f6cbb67b2bdf (table default.Table_3b5a67572c914054908f9bde6bb0ee09 [id=95f51150591e45e8a69bfacc9ac17f24]) successfully deleted
I20260618 13:59:20.706256 28800 raft_consensus.cc:2243] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.706485 28800 raft_consensus.cc:2272] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.707116 28800 ts_tablet_manager.cc:1916] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.713201 28797 tablet_bootstrap.cc:492] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:20.713359 28797 ts_tablet_manager.cc:1403] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.045s user 0.017s sys 0.004s
I20260618 13:59:20.713596 28800 ts_tablet_manager.cc:1929] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.714154 28800 log.cc:1199] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/f7027a4138784d029255fc89c73104a3
I20260618 13:59:20.714224 28797 raft_consensus.cc:359] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.714486 28797 raft_consensus.cc:385] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.714596 28797 raft_consensus.cc:740] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.714658 28800 ts_tablet_manager.cc:1950] T f7027a4138784d029255fc89c73104a3 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:20.714824 28797 consensus_queue.cc:260] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.715294 28797 ts_tablet_manager.cc:1434] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:20.715559 28797 tablet_bootstrap.cc:492] T 59613383d2294f9ab6bce6321aad44bb P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:20.717223 28797 tablet_bootstrap.cc:654] T 59613383d2294f9ab6bce6321aad44bb P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.717243 28801 ts_tablet_manager.cc:1929] T f7027a4138784d029255fc89c73104a3 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.717458 28801 log.cc:1199] T f7027a4138784d029255fc89c73104a3 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/f7027a4138784d029255fc89c73104a3
I20260618 13:59:20.717859 28801 ts_tablet_manager.cc:1950] T f7027a4138784d029255fc89c73104a3 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:20.718945 28800 tablet_replica.cc:333] T a6189fed26fb471c9724fd594054a58c P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:20.719118 28800 raft_consensus.cc:2243] T a6189fed26fb471c9724fd594054a58c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.719293 28800 raft_consensus.cc:2272] T a6189fed26fb471c9724fd594054a58c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.720067 28800 ts_tablet_manager.cc:1916] T a6189fed26fb471c9724fd594054a58c P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.720482 28797 tablet_bootstrap.cc:492] T 59613383d2294f9ab6bce6321aad44bb P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:20.720660 28797 ts_tablet_manager.cc:1403] T 59613383d2294f9ab6bce6321aad44bb P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260618 13:59:20.720829 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet a6189fed26fb471c9724fd594054a58c (table default.Table_4ac6ae42a7f94b79abb170afc27cd978 [id=979317e20c204b109fb05c4128c0e2a3]) successfully deleted
W20260618 13:59:20.721125 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d6e3060118e844559803a649224d7474 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d6e3060118e844559803a649224d7474 already in progress: creating tablet
W20260618 13:59:20.721336 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet f4447a3e87bd476e84f59dd4c4185b26 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f4447a3e87bd476e84f59dd4c4185b26 already in progress: creating tablet
I20260618 13:59:20.721418 28797 raft_consensus.cc:359] T 59613383d2294f9ab6bce6321aad44bb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.721529 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet f7027a4138784d029255fc89c73104a3 (table default.Table_7559b439752d4fa68c4fb6afee136bfc [id=61b2edf42a374060934bf910c3f2dde5]) successfully deleted
I20260618 13:59:20.721614 28797 raft_consensus.cc:385] T 59613383d2294f9ab6bce6321aad44bb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.721695 28797 raft_consensus.cc:740] T 59613383d2294f9ab6bce6321aad44bb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
W20260618 13:59:20.721740 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet f4447a3e87bd476e84f59dd4c4185b26 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f4447a3e87bd476e84f59dd4c4185b26 already in progress: creating tablet
W20260618 13:59:20.721966 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d6e3060118e844559803a649224d7474 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d6e3060118e844559803a649224d7474 already in progress: creating tablet
I20260618 13:59:20.721928 28797 consensus_queue.cc:260] T 59613383d2294f9ab6bce6321aad44bb P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.722147 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet f7027a4138784d029255fc89c73104a3 (table default.Table_7559b439752d4fa68c4fb6afee136bfc [id=61b2edf42a374060934bf910c3f2dde5]) successfully deleted
I20260618 13:59:20.722400 28797 ts_tablet_manager.cc:1434] T 59613383d2294f9ab6bce6321aad44bb P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:20.722733 28797 tablet_bootstrap.cc:492] T 07a4e95bd6ae47beac34fd36d6e956e3 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:20.723548 28798 tablet_bootstrap.cc:492] T 59613383d2294f9ab6bce6321aad44bb P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:20.723701 28798 ts_tablet_manager.cc:1403] T 59613383d2294f9ab6bce6321aad44bb P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.042s user 0.001s sys 0.002s
I20260618 13:59:20.724352 28797 tablet_bootstrap.cc:654] T 07a4e95bd6ae47beac34fd36d6e956e3 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.724488 28798 raft_consensus.cc:359] T 59613383d2294f9ab6bce6321aad44bb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.724663 28798 raft_consensus.cc:385] T 59613383d2294f9ab6bce6321aad44bb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.724759 28798 raft_consensus.cc:740] T 59613383d2294f9ab6bce6321aad44bb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.724972 28798 consensus_queue.cc:260] T 59613383d2294f9ab6bce6321aad44bb P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.725445 28798 ts_tablet_manager.cc:1434] T 59613383d2294f9ab6bce6321aad44bb P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:20.725723 28798 tablet_bootstrap.cc:492] T 07a4e95bd6ae47beac34fd36d6e956e3 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:20.726112 28797 tablet_bootstrap.cc:492] T 07a4e95bd6ae47beac34fd36d6e956e3 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:20.726238 28797 ts_tablet_manager.cc:1403] T 07a4e95bd6ae47beac34fd36d6e956e3 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260618 13:59:20.726920 28797 raft_consensus.cc:359] T 07a4e95bd6ae47beac34fd36d6e956e3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.727098 28797 raft_consensus.cc:385] T 07a4e95bd6ae47beac34fd36d6e956e3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.727192 28797 raft_consensus.cc:740] T 07a4e95bd6ae47beac34fd36d6e956e3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.727420 28798 tablet_bootstrap.cc:654] T 07a4e95bd6ae47beac34fd36d6e956e3 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.727448 28797 consensus_queue.cc:260] T 07a4e95bd6ae47beac34fd36d6e956e3 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.728081 28797 ts_tablet_manager.cc:1434] T 07a4e95bd6ae47beac34fd36d6e956e3 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:20.728339 28797 tablet_bootstrap.cc:492] T 587b82a6da8d4fe48b26070803831d9d P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:20.729962 28797 tablet_bootstrap.cc:654] T 587b82a6da8d4fe48b26070803831d9d P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.734118 28799 tablet_bootstrap.cc:492] T 587b82a6da8d4fe48b26070803831d9d P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:20.734283 28799 ts_tablet_manager.cc:1403] T 587b82a6da8d4fe48b26070803831d9d P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.062s user 0.003s sys 0.019s
I20260618 13:59:20.735148 28799 raft_consensus.cc:359] T 587b82a6da8d4fe48b26070803831d9d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.735360 28799 raft_consensus.cc:385] T 587b82a6da8d4fe48b26070803831d9d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.735464 28799 raft_consensus.cc:740] T 587b82a6da8d4fe48b26070803831d9d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.735646 28800 ts_tablet_manager.cc:1929] T a6189fed26fb471c9724fd594054a58c P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.735713 28799 consensus_queue.cc:260] T 587b82a6da8d4fe48b26070803831d9d P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.735863 28227 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_118ec6b705604f45a7d091a960a74735" } new_table_name: "default.Table_7d3c22c138704d71a7f2a0e756cfa3a2" modify_external_catalogs: true
I20260618 13:59:20.735862 28800 log.cc:1199] T a6189fed26fb471c9724fd594054a58c P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/a6189fed26fb471c9724fd594054a58c
I20260618 13:59:20.736230 28799 ts_tablet_manager.cc:1434] T 587b82a6da8d4fe48b26070803831d9d P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:20.736413 28800 ts_tablet_manager.cc:1950] T a6189fed26fb471c9724fd594054a58c P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:20.736512 28799 tablet_bootstrap.cc:492] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:20.737663 28224 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 144aa8984a44410fb3f90dc99e9b2274 consensus state has no leader
I20260618 13:59:20.738260 28797 tablet_bootstrap.cc:492] T 587b82a6da8d4fe48b26070803831d9d P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:20.738459 28797 ts_tablet_manager.cc:1403] T 587b82a6da8d4fe48b26070803831d9d P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260618 13:59:20.739281 28797 raft_consensus.cc:359] T 587b82a6da8d4fe48b26070803831d9d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.739519 28797 raft_consensus.cc:385] T 587b82a6da8d4fe48b26070803831d9d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.739614 28797 raft_consensus.cc:740] T 587b82a6da8d4fe48b26070803831d9d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.739888 28797 consensus_queue.cc:260] T 587b82a6da8d4fe48b26070803831d9d P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:20.740432 28226 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 311ba4dd83984f379c8f4f8274bc26c1 consensus state has no leader
I20260618 13:59:20.741674 28797 ts_tablet_manager.cc:1434] T 587b82a6da8d4fe48b26070803831d9d P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:20.743335 28799 tablet_bootstrap.cc:654] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.743388 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 587b82a6da8d4fe48b26070803831d9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:20.743577 28797 tablet_bootstrap.cc:492] T 91c250dfbd72416d9d74f6cbb67b2bdf P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:20.744575 28221 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7e5517e820ae47de810ff817f5382839 consensus state has no leader
I20260618 13:59:20.745239 28797 tablet_bootstrap.cc:654] T 91c250dfbd72416d9d74f6cbb67b2bdf P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.745198 28228 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 397c7abe26f4451298558d6a0875b677 (table default.Table_9fe2f476cd334e84adf5b413afda9dcc [id=ce780967c12a412a8d7f0bef2f9c00e3]). Expected version 4 got 0
W20260618 13:59:20.745608 28225 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 397c7abe26f4451298558d6a0875b677 consensus state has no leader
I20260618 13:59:20.749140 28802 tablet_replica.cc:333] T 587b82a6da8d4fe48b26070803831d9d P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:20.749358 28802 raft_consensus.cc:2243] T 587b82a6da8d4fe48b26070803831d9d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.749732 28802 raft_consensus.cc:2272] T 587b82a6da8d4fe48b26070803831d9d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.750926 28802 ts_tablet_manager.cc:1916] T 587b82a6da8d4fe48b26070803831d9d P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.753474 28797 tablet_bootstrap.cc:492] T 91c250dfbd72416d9d74f6cbb67b2bdf P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:20.753634 28797 ts_tablet_manager.cc:1403] T 91c250dfbd72416d9d74f6cbb67b2bdf P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260618 13:59:20.754408 28797 raft_consensus.cc:359] T 91c250dfbd72416d9d74f6cbb67b2bdf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.754551 28389 tablet_service.cc:1511] Processing CreateTablet for tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 (DEFAULT_TABLE table=default.Table_ceb33d60d86e4fd898ae67c78da94a55 [id=3251ae257e0f49b99352dcc1ef004588]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.754662 28797 raft_consensus.cc:385] T 91c250dfbd72416d9d74f6cbb67b2bdf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.754752 28797 raft_consensus.cc:740] T 91c250dfbd72416d9d74f6cbb67b2bdf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.754936 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.754999 28797 consensus_queue.cc:260] T 91c250dfbd72416d9d74f6cbb67b2bdf P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.757418 28797 ts_tablet_manager.cc:1434] T 91c250dfbd72416d9d74f6cbb67b2bdf P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:20.757864 28797 tablet_bootstrap.cc:492] T f4447a3e87bd476e84f59dd4c4185b26 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:20.761523 28802 ts_tablet_manager.cc:1929] T 587b82a6da8d4fe48b26070803831d9d P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.762100 28797 tablet_bootstrap.cc:654] T f4447a3e87bd476e84f59dd4c4185b26 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.762835 28802 log.cc:1199] T 587b82a6da8d4fe48b26070803831d9d P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/587b82a6da8d4fe48b26070803831d9d
I20260618 13:59:20.763418 28802 ts_tablet_manager.cc:1950] T 587b82a6da8d4fe48b26070803831d9d P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:20.764820 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 587b82a6da8d4fe48b26070803831d9d (table default.Table_e7787f99590e428ca58ae371cdfeffd5 [id=262236d66ee24ad8a9e4e27b4b1c3ac1]) successfully deleted
I20260618 13:59:20.768966 28799 tablet_bootstrap.cc:492] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:20.768926 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet 59613383d2294f9ab6bce6321aad44bb with delete_type TABLET_DATA_DELETED (replaced by tablet 29bf263d20934ed6a6efb4b215252c00) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.769212 28799 ts_tablet_manager.cc:1403] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.007s
I20260618 13:59:20.769295 28801 tablet_replica.cc:333] T 59613383d2294f9ab6bce6321aad44bb P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:20.769520 28801 raft_consensus.cc:2243] T 59613383d2294f9ab6bce6321aad44bb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.769730 28801 raft_consensus.cc:2272] T 59613383d2294f9ab6bce6321aad44bb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.770145 28799 raft_consensus.cc:359] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.770349 28799 raft_consensus.cc:385] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.770447 28799 raft_consensus.cc:740] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.770699 28799 consensus_queue.cc:260] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.770797 28801 ts_tablet_manager.cc:1916] T 59613383d2294f9ab6bce6321aad44bb P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.772235 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet 59613383d2294f9ab6bce6321aad44bb with delete_type TABLET_DATA_DELETED (replaced by tablet 29bf263d20934ed6a6efb4b215252c00) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.772513 28526 tablet_service.cc:1511] Processing CreateTablet for tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 (DEFAULT_TABLE table=default.Table_ceb33d60d86e4fd898ae67c78da94a55 [id=3251ae257e0f49b99352dcc1ef004588]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.772683 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet 07a4e95bd6ae47beac34fd36d6e956e3 with delete_type TABLET_DATA_DELETED (replaced by tablet eeffc96b9b97441d9301e4919d0047e1) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.772951 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.773984 28800 tablet_replica.cc:333] T 59613383d2294f9ab6bce6321aad44bb P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:20.774192 28800 raft_consensus.cc:2243] T 59613383d2294f9ab6bce6321aad44bb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.774366 28800 raft_consensus.cc:2272] T 59613383d2294f9ab6bce6321aad44bb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.774592 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 587b82a6da8d4fe48b26070803831d9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.777544 28800 ts_tablet_manager.cc:1916] T 59613383d2294f9ab6bce6321aad44bb P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.780318 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet 91c250dfbd72416d9d74f6cbb67b2bdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.780390 28655 tablet_service.cc:1511] Processing CreateTablet for tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 (DEFAULT_TABLE table=default.Table_ceb33d60d86e4fd898ae67c78da94a55 [id=3251ae257e0f49b99352dcc1ef004588]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.780874 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.784279 28800 ts_tablet_manager.cc:1929] T 59613383d2294f9ab6bce6321aad44bb P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.784441 28800 log.cc:1199] T 59613383d2294f9ab6bce6321aad44bb P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/59613383d2294f9ab6bce6321aad44bb
I20260618 13:59:20.784859 28800 ts_tablet_manager.cc:1950] T 59613383d2294f9ab6bce6321aad44bb P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:20.786036 28799 ts_tablet_manager.cc:1434] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.017s user 0.000s sys 0.002s
I20260618 13:59:20.788669 28799 tablet_bootstrap.cc:492] T ee783d2c7e274158b3de2cc186f50903 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:20.790302 28799 tablet_bootstrap.cc:654] T ee783d2c7e274158b3de2cc186f50903 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.791131 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet a6189fed26fb471c9724fd594054a58c (table default.Table_4ac6ae42a7f94b79abb170afc27cd978 [id=979317e20c204b109fb05c4128c0e2a3]) successfully deleted
I20260618 13:59:20.791365 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 59613383d2294f9ab6bce6321aad44bb (table default.Table_58d6a901ec6a4741b232e4d1fe515cb5 [id=7923680bde144e3dbb78c922d09540b9]) successfully deleted
I20260618 13:59:20.791676 28800 tablet_replica.cc:333] T 07a4e95bd6ae47beac34fd36d6e956e3 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:20.791150 28801 ts_tablet_manager.cc:1929] T 59613383d2294f9ab6bce6321aad44bb P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.791865 28800 raft_consensus.cc:2243] T 07a4e95bd6ae47beac34fd36d6e956e3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.792003 28801 log.cc:1199] T 59613383d2294f9ab6bce6321aad44bb P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/59613383d2294f9ab6bce6321aad44bb
I20260618 13:59:20.792047 28800 raft_consensus.cc:2272] T 07a4e95bd6ae47beac34fd36d6e956e3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.792775 28800 ts_tablet_manager.cc:1916] T 07a4e95bd6ae47beac34fd36d6e956e3 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.793237 28801 ts_tablet_manager.cc:1950] T 59613383d2294f9ab6bce6321aad44bb P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:20.794339 28220 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet b1f734a3f29b42e591d5564e99292b5e
I20260618 13:59:20.794981 28220 catalog_manager.cc:6784] ReplaceTablet: tablet b1f734a3f29b42e591d5564e99292b5e deleted and replaced by tablet e025b64380314546af8a1ab90c817825
I20260618 13:59:20.795431 28229 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet d5cb3137659a4481840fe04174ef682d
I20260618 13:59:20.795941 28229 catalog_manager.cc:6784] ReplaceTablet: tablet d5cb3137659a4481840fe04174ef682d deleted and replaced by tablet 68b8293a7c924a449307ab88a1cf9290
I20260618 13:59:20.797159 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet d5cb3137659a4481840fe04174ef682d with delete_type TABLET_DATA_DELETED (replaced by tablet 68b8293a7c924a449307ab88a1cf9290) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:20.797842 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet d5cb3137659a4481840fe04174ef682d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d5cb3137659a4481840fe04174ef682d already in progress: creating tablet
W20260618 13:59:20.797953 28227 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 570d36a27b7d417989ad72885d38ff63 consensus state has no leader
I20260618 13:59:20.798494 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet b1f734a3f29b42e591d5564e99292b5e with delete_type TABLET_DATA_DELETED (replaced by tablet e025b64380314546af8a1ab90c817825) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:20.798373 28800 ts_tablet_manager.cc:1929] T 07a4e95bd6ae47beac34fd36d6e956e3 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.799232 28800 log.cc:1199] T 07a4e95bd6ae47beac34fd36d6e956e3 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/07a4e95bd6ae47beac34fd36d6e956e3
W20260618 13:59:20.799295 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet b1f734a3f29b42e591d5564e99292b5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b1f734a3f29b42e591d5564e99292b5e already in progress: creating tablet
I20260618 13:59:20.799711 28800 ts_tablet_manager.cc:1950] T 07a4e95bd6ae47beac34fd36d6e956e3 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:20.799963 28797 tablet_bootstrap.cc:492] T f4447a3e87bd476e84f59dd4c4185b26 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:20.800129 28797 ts_tablet_manager.cc:1403] T f4447a3e87bd476e84f59dd4c4185b26 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.042s user 0.015s sys 0.017s
I20260618 13:59:20.800860 28800 tablet_replica.cc:333] T 587b82a6da8d4fe48b26070803831d9d P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:20.800904 28797 raft_consensus.cc:359] T f4447a3e87bd476e84f59dd4c4185b26 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.801100 28797 raft_consensus.cc:385] T f4447a3e87bd476e84f59dd4c4185b26 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.801177 28797 raft_consensus.cc:740] T f4447a3e87bd476e84f59dd4c4185b26 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.801216 28800 raft_consensus.cc:2243] T 587b82a6da8d4fe48b26070803831d9d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.801492 28800 raft_consensus.cc:2272] T 587b82a6da8d4fe48b26070803831d9d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.801436 28797 consensus_queue.cc:260] T f4447a3e87bd476e84f59dd4c4185b26 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.803450 28797 ts_tablet_manager.cc:1434] T f4447a3e87bd476e84f59dd4c4185b26 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:20.803800 28797 tablet_bootstrap.cc:492] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:20.805842 28797 tablet_bootstrap.cc:654] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.807535 28800 ts_tablet_manager.cc:1916] T 587b82a6da8d4fe48b26070803831d9d P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.811025 28227 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_f32d5142bcf64f5c9e94b015dd620e10" } modify_external_catalogs: true
I20260618 13:59:20.811342 28227 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_f32d5142bcf64f5c9e94b015dd620e10" } modify_external_catalogs: true
I20260618 13:59:20.812137 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 59613383d2294f9ab6bce6321aad44bb (table default.Table_58d6a901ec6a4741b232e4d1fe515cb5 [id=7923680bde144e3dbb78c922d09540b9]) successfully deleted
I20260618 13:59:20.814034 28800 ts_tablet_manager.cc:1929] T 587b82a6da8d4fe48b26070803831d9d P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.814185 28800 log.cc:1199] T 587b82a6da8d4fe48b26070803831d9d P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/587b82a6da8d4fe48b26070803831d9d
I20260618 13:59:20.814559 28800 ts_tablet_manager.cc:1950] T 587b82a6da8d4fe48b26070803831d9d P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:20.814678 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet b1f734a3f29b42e591d5564e99292b5e with delete_type TABLET_DATA_DELETED (replaced by tablet e025b64380314546af8a1ab90c817825) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.815825 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet b1f734a3f29b42e591d5564e99292b5e with delete_type TABLET_DATA_DELETED (replaced by tablet e025b64380314546af8a1ab90c817825) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.816084 28800 tablet_replica.cc:333] T 91c250dfbd72416d9d74f6cbb67b2bdf P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:20.816248 28800 raft_consensus.cc:2243] T 91c250dfbd72416d9d74f6cbb67b2bdf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.816362 28797 tablet_bootstrap.cc:492] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:20.816434 28800 raft_consensus.cc:2272] T 91c250dfbd72416d9d74f6cbb67b2bdf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.816522 28797 ts_tablet_manager.cc:1403] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.001s
I20260618 13:59:20.816535 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet d5cb3137659a4481840fe04174ef682d with delete_type TABLET_DATA_DELETED (replaced by tablet 68b8293a7c924a449307ab88a1cf9290) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.817135 28800 ts_tablet_manager.cc:1916] T 91c250dfbd72416d9d74f6cbb67b2bdf P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.817396 28797 raft_consensus.cc:359] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.817620 28797 raft_consensus.cc:385] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.817731 28797 raft_consensus.cc:740] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.817752 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet d5cb3137659a4481840fe04174ef682d with delete_type TABLET_DATA_DELETED (replaced by tablet 68b8293a7c924a449307ab88a1cf9290) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.817967 28797 consensus_queue.cc:260] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.818513 28797 ts_tablet_manager.cc:1434] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260618 13:59:20.818814 28797 tablet_bootstrap.cc:492] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:20.820473 28797 tablet_bootstrap.cc:654] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.822156 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet f4447a3e87bd476e84f59dd4c4185b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.822167 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 07a4e95bd6ae47beac34fd36d6e956e3 (table default.Table_f0607bf6529e4f209e00f49908420141 [id=972bc2c124ac459da06ec4093c5b564b]) successfully deleted
I20260618 13:59:20.822381 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 587b82a6da8d4fe48b26070803831d9d (table default.Table_e7787f99590e428ca58ae371cdfeffd5 [id=262236d66ee24ad8a9e4e27b4b1c3ac1]) successfully deleted
I20260618 13:59:20.825762 28798 tablet_bootstrap.cc:492] T 07a4e95bd6ae47beac34fd36d6e956e3 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:20.826001 28798 ts_tablet_manager.cc:1403] T 07a4e95bd6ae47beac34fd36d6e956e3 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.100s user 0.003s sys 0.000s
I20260618 13:59:20.827049 28798 raft_consensus.cc:359] T 07a4e95bd6ae47beac34fd36d6e956e3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.827306 28798 raft_consensus.cc:385] T 07a4e95bd6ae47beac34fd36d6e956e3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.827396 28798 raft_consensus.cc:740] T 07a4e95bd6ae47beac34fd36d6e956e3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.827725 28798 consensus_queue.cc:260] T 07a4e95bd6ae47beac34fd36d6e956e3 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.830664 28800 ts_tablet_manager.cc:1929] T 91c250dfbd72416d9d74f6cbb67b2bdf P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.832669 28797 tablet_bootstrap.cc:492] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:20.832939 28797 ts_tablet_manager.cc:1403] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.003s
I20260618 13:59:20.833491 28798 ts_tablet_manager.cc:1434] T 07a4e95bd6ae47beac34fd36d6e956e3 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260618 13:59:20.833885 28229 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_c1cda1e770c24c7eb9ab6c1ad8a7e786" } modify_external_catalogs: true
I20260618 13:59:20.834077 28229 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_c1cda1e770c24c7eb9ab6c1ad8a7e786" } modify_external_catalogs: true
I20260618 13:59:20.834249 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.834575 28797 raft_consensus.cc:359] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.834872 28797 raft_consensus.cc:385] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.835005 28797 raft_consensus.cc:740] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.835341 28797 consensus_queue.cc:260] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.838225 28797 ts_tablet_manager.cc:1434] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260618 13:59:20.839546 28797 tablet_bootstrap.cc:492] T ee783d2c7e274158b3de2cc186f50903 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:20.841352 28797 tablet_bootstrap.cc:654] T ee783d2c7e274158b3de2cc186f50903 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.842254 28800 log.cc:1199] T 91c250dfbd72416d9d74f6cbb67b2bdf P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/91c250dfbd72416d9d74f6cbb67b2bdf
I20260618 13:59:20.842720 28800 ts_tablet_manager.cc:1950] T 91c250dfbd72416d9d74f6cbb67b2bdf P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:20.844308 28800 tablet_replica.cc:333] T f4447a3e87bd476e84f59dd4c4185b26 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:20.844539 28800 raft_consensus.cc:2243] T f4447a3e87bd476e84f59dd4c4185b26 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.844758 28800 raft_consensus.cc:2272] T f4447a3e87bd476e84f59dd4c4185b26 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.845404 28800 ts_tablet_manager.cc:1916] T f4447a3e87bd476e84f59dd4c4185b26 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.845659 28798 tablet_bootstrap.cc:492] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:20.846817 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet 07a4e95bd6ae47beac34fd36d6e956e3 with delete_type TABLET_DATA_DELETED (replaced by tablet eeffc96b9b97441d9301e4919d0047e1) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:20.847179 28801 tablet_replica.cc:333] T 07a4e95bd6ae47beac34fd36d6e956e3 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:20.847364 28801 raft_consensus.cc:2243] T 07a4e95bd6ae47beac34fd36d6e956e3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.847545 28801 raft_consensus.cc:2272] T 07a4e95bd6ae47beac34fd36d6e956e3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.848591 28801 ts_tablet_manager.cc:1916] T 07a4e95bd6ae47beac34fd36d6e956e3 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.852828 28800 ts_tablet_manager.cc:1929] T f4447a3e87bd476e84f59dd4c4185b26 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.852985 28800 log.cc:1199] T f4447a3e87bd476e84f59dd4c4185b26 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/f4447a3e87bd476e84f59dd4c4185b26
I20260618 13:59:20.853339 28800 ts_tablet_manager.cc:1950] T f4447a3e87bd476e84f59dd4c4185b26 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:20.857563 28801 ts_tablet_manager.cc:1929] T 07a4e95bd6ae47beac34fd36d6e956e3 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.857631 28797 tablet_bootstrap.cc:492] T ee783d2c7e274158b3de2cc186f50903 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:20.857777 28797 ts_tablet_manager.cc:1403] T ee783d2c7e274158b3de2cc186f50903 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.003s
I20260618 13:59:20.858588 28797 raft_consensus.cc:359] T ee783d2c7e274158b3de2cc186f50903 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.858791 28797 raft_consensus.cc:385] T ee783d2c7e274158b3de2cc186f50903 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.858878 28797 raft_consensus.cc:740] T ee783d2c7e274158b3de2cc186f50903 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.859155 28797 consensus_queue.cc:260] T ee783d2c7e274158b3de2cc186f50903 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:20.859464 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet b1f734a3f29b42e591d5564e99292b5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b1f734a3f29b42e591d5564e99292b5e already in progress: creating tablet
W20260618 13:59:20.859877 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d5cb3137659a4481840fe04174ef682d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d5cb3137659a4481840fe04174ef682d already in progress: creating tablet
I20260618 13:59:20.863112 28797 ts_tablet_manager.cc:1434] T ee783d2c7e274158b3de2cc186f50903 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260618 13:59:20.864444 28801 log.cc:1199] T 07a4e95bd6ae47beac34fd36d6e956e3 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/07a4e95bd6ae47beac34fd36d6e956e3
I20260618 13:59:20.866348 28797 tablet_bootstrap.cc:492] T 8c9d89fa75174a969ded07e0596db99b P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:20.868080 28797 tablet_bootstrap.cc:654] T 8c9d89fa75174a969ded07e0596db99b P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.869041 28800 tablet_replica.cc:333] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:20.869225 28800 raft_consensus.cc:2243] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.869400 28800 raft_consensus.cc:2272] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.869555 28801 ts_tablet_manager.cc:1950] T 07a4e95bd6ae47beac34fd36d6e956e3 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:20.869397 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 91c250dfbd72416d9d74f6cbb67b2bdf (table default.Table_3b5a67572c914054908f9bde6bb0ee09 [id=95f51150591e45e8a69bfacc9ac17f24]) successfully deleted
I20260618 13:59:20.869972 28800 ts_tablet_manager.cc:1916] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:20.869999 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet b1f734a3f29b42e591d5564e99292b5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b1f734a3f29b42e591d5564e99292b5e already in progress: creating tablet
W20260618 13:59:20.870795 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d5cb3137659a4481840fe04174ef682d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d5cb3137659a4481840fe04174ef682d already in progress: creating tablet
I20260618 13:59:20.872043 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet f4447a3e87bd476e84f59dd4c4185b26 (table default.Table_4cf6e1e03c724f42a0d15aed65142720 [id=80246ec4fbca44248528c2c379034cee]) successfully deleted
I20260618 13:59:20.861644 28798 tablet_bootstrap.cc:654] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.873507 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 07a4e95bd6ae47beac34fd36d6e956e3 (table default.Table_f0607bf6529e4f209e00f49908420141 [id=972bc2c124ac459da06ec4093c5b564b]) successfully deleted
I20260618 13:59:20.876487 28228 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 717c38fbd9cb492b9547372e2160f84f (table default.Table_a2f92d4bf5bc49aca105a2886053d230 [id=3bcc877eddec4d00b3dc1760cc635b36]). Expected version 5 got 0
I20260618 13:59:20.878261 28220 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 717c38fbd9cb492b9547372e2160f84f (table default.Table_a2f92d4bf5bc49aca105a2886053d230 [id=3bcc877eddec4d00b3dc1760cc635b36]). Expected version 5 got 0
I20260618 13:59:20.880941 28228 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 717c38fbd9cb492b9547372e2160f84f (table default.Table_a2f92d4bf5bc49aca105a2886053d230 [id=3bcc877eddec4d00b3dc1760cc635b36]). Expected version 5 got 0
I20260618 13:59:20.884128 28228 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 717c38fbd9cb492b9547372e2160f84f (table default.Table_a2f92d4bf5bc49aca105a2886053d230 [id=3bcc877eddec4d00b3dc1760cc635b36]). Expected version 5 got 0
I20260618 13:59:20.885483 28800 ts_tablet_manager.cc:1929] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.885639 28800 log.cc:1199] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/9fd3fbcf0d9745a1b3d3408f95022bf6
I20260618 13:59:20.885998 28800 ts_tablet_manager.cc:1950] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:20.887931 28797 tablet_bootstrap.cc:492] T 8c9d89fa75174a969ded07e0596db99b P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:20.888096 28797 ts_tablet_manager.cc:1403] T 8c9d89fa75174a969ded07e0596db99b P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.000s
I20260618 13:59:20.888876 28797 raft_consensus.cc:359] T 8c9d89fa75174a969ded07e0596db99b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:20.889101 28797 raft_consensus.cc:385] T 8c9d89fa75174a969ded07e0596db99b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.889173 28797 raft_consensus.cc:740] T 8c9d89fa75174a969ded07e0596db99b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.889447 28797 consensus_queue.cc:260] T 8c9d89fa75174a969ded07e0596db99b P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:20.889609 28389 tablet_service.cc:1511] Processing CreateTablet for tablet ba0d05ea44e1473fb4b24f61eab096e4 (DEFAULT_TABLE table=default.Table_1de2d9113caa4693a242a5ec6f6ee0c1 [id=7d40d815ea4544aea591ca23b96b5dd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.890161 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba0d05ea44e1473fb4b24f61eab096e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.893438 28797 ts_tablet_manager.cc:1434] T 8c9d89fa75174a969ded07e0596db99b P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260618 13:59:20.898293 28797 tablet_bootstrap.cc:492] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:20.899968 28797 tablet_bootstrap.cc:654] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.904412 28391 tablet_service.cc:1511] Processing CreateTablet for tablet 8be4374ae7cf4907940ae90299bf9ba0 (DEFAULT_TABLE table=default.Table_0a13b90182654346acbbc8f47a56436d [id=c31d65adb55f458c9ced14d598464f89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.904999 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8be4374ae7cf4907940ae90299bf9ba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.907747 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet ee783d2c7e274158b3de2cc186f50903 with delete_type TABLET_DATA_DELETED (replaced by tablet 311ba4dd83984f379c8f4f8274bc26c1) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.908108 28800 tablet_replica.cc:333] T ee783d2c7e274158b3de2cc186f50903 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:20.908144 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c9d89fa75174a969ded07e0596db99b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:20.908326 28800 raft_consensus.cc:2243] T ee783d2c7e274158b3de2cc186f50903 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.908536 28797 tablet_bootstrap.cc:492] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:20.908541 28800 raft_consensus.cc:2272] T ee783d2c7e274158b3de2cc186f50903 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.908684 28797 ts_tablet_manager.cc:1403] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20260618 13:59:20.909356 28800 ts_tablet_manager.cc:1916] T ee783d2c7e274158b3de2cc186f50903 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.909554 28797 raft_consensus.cc:359] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:20.910349 28797 raft_consensus.cc:385] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.910470 28797 raft_consensus.cc:740] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.910774 28797 consensus_queue.cc:260] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:20.911413 28797 ts_tablet_manager.cc:1434] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260618 13:59:20.911792 28797 tablet_bootstrap.cc:492] T a2ca1e22c38e49d793d413e9eaaac06f P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:20.913482 28797 tablet_bootstrap.cc:654] T a2ca1e22c38e49d793d413e9eaaac06f P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.917761 28800 ts_tablet_manager.cc:1929] T ee783d2c7e274158b3de2cc186f50903 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.917882 28800 log.cc:1199] T ee783d2c7e274158b3de2cc186f50903 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/ee783d2c7e274158b3de2cc186f50903
I20260618 13:59:20.918227 28800 ts_tablet_manager.cc:1950] T ee783d2c7e274158b3de2cc186f50903 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:20.919988 28228 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 570d36a27b7d417989ad72885d38ff63 (table default.Table_7d3c22c138704d71a7f2a0e756cfa3a2 [id=e38a333ca62d42518ab53a9ac17df2b7]). Expected version 3 got 0
I20260618 13:59:20.922631 28228 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 570d36a27b7d417989ad72885d38ff63 (table default.Table_7d3c22c138704d71a7f2a0e756cfa3a2 [id=e38a333ca62d42518ab53a9ac17df2b7]). Expected version 3 got 0
I20260618 13:59:20.925163 28800 tablet_replica.cc:333] T 8c9d89fa75174a969ded07e0596db99b P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:20.925395 28800 raft_consensus.cc:2243] T 8c9d89fa75174a969ded07e0596db99b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:20.925621 28800 raft_consensus.cc:2272] T 8c9d89fa75174a969ded07e0596db99b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:20.928390 28526 tablet_service.cc:1511] Processing CreateTablet for tablet ba0d05ea44e1473fb4b24f61eab096e4 (DEFAULT_TABLE table=default.Table_1de2d9113caa4693a242a5ec6f6ee0c1 [id=7d40d815ea4544aea591ca23b96b5dd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.928889 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba0d05ea44e1473fb4b24f61eab096e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.931433 28798 tablet_bootstrap.cc:492] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:20.931599 28798 ts_tablet_manager.cc:1403] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.086s user 0.002s sys 0.002s
I20260618 13:59:20.931643 28655 tablet_service.cc:1511] Processing CreateTablet for tablet ba0d05ea44e1473fb4b24f61eab096e4 (DEFAULT_TABLE table=default.Table_1de2d9113caa4693a242a5ec6f6ee0c1 [id=7d40d815ea4544aea591ca23b96b5dd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.932129 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba0d05ea44e1473fb4b24f61eab096e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.932610 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 (table default.Table_e10ecf3a190148d0b17f97204f155353 [id=179292b8d0574f7bb5b6edb5dcad5cb9]) successfully deleted
I20260618 13:59:20.932803 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet ee783d2c7e274158b3de2cc186f50903 (table default.Table_dd833764ce1041cba71575be46d771a9 [id=d8ef984c664f4a6598c70a48e9aff826]) successfully deleted
I20260618 13:59:20.933279 28228 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_56009fb08789451f82400e86054dd066" } modify_external_catalogs: true
I20260618 13:59:20.933394 28800 ts_tablet_manager.cc:1916] T 8c9d89fa75174a969ded07e0596db99b P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:20.933451 28228 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_56009fb08789451f82400e86054dd066" } modify_external_catalogs: true
I20260618 13:59:20.935346 28386 tablet_service.cc:1511] Processing CreateTablet for tablet 95db5ddaf5d5486988750efed5972eb6 (DEFAULT_TABLE table=default.Table_067483ebd76c49e595543cc25ff1db56 [id=5dadba9b472745259796cf9015e88c65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.935446 28387 tablet_service.cc:1511] Processing CreateTablet for tablet bab50e1f17a74eee8cad942525002996 (DEFAULT_TABLE table=default.Table_8f22977cfa824da99508b4063a05f8e6 [id=b527fd0afd76403f94cb5fe9f4c95094]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.935735 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95db5ddaf5d5486988750efed5972eb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.936021 28225 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_1fd86f80c8fc4490aaadc1278d20df3a" } modify_external_catalogs: true
I20260618 13:59:20.936221 28225 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_1fd86f80c8fc4490aaadc1278d20df3a" } modify_external_catalogs: true
I20260618 13:59:20.937969 28387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bab50e1f17a74eee8cad942525002996. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.939496 28800 ts_tablet_manager.cc:1929] T 8c9d89fa75174a969ded07e0596db99b P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:20.939445 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_7700ee6bae434a01b621ad02088cc5f0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:20.939618 28800 log.cc:1199] T 8c9d89fa75174a969ded07e0596db99b P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/8c9d89fa75174a969ded07e0596db99b
I20260618 13:59:20.940023 28800 ts_tablet_manager.cc:1950] T 8c9d89fa75174a969ded07e0596db99b P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
W20260618 13:59:20.940083 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7700ee6bae434a01b621ad02088cc5f0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.941066 28798 raft_consensus.cc:359] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.941327 28798 raft_consensus.cc:385] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.941406 28798 raft_consensus.cc:740] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.941499 28221 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_2bf9616ba425458d8c7a4e9a6854d122" } modify_external_catalogs: true
I20260618 13:59:20.941708 28221 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_2bf9616ba425458d8c7a4e9a6854d122" } modify_external_catalogs: true
I20260618 13:59:20.942385 28650 tablet_service.cc:1511] Processing CreateTablet for tablet 5df99b28e9ae4b6bbad0fe0dafd125d6 (DEFAULT_TABLE table=default.Table_2bf9616ba425458d8c7a4e9a6854d122 [id=e3f4e13202da42a786d3376ba662b31a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.942565 28226 master_service.cc:960] ReplaceTablet: received request to replace tablet b739fa1847184290833e13a2515f3a3a from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:20.942785 28650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5df99b28e9ae4b6bbad0fe0dafd125d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.943917 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_696ead97d50747228517c265538b25fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:20.944546 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_696ead97d50747228517c265538b25fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.945887 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_0827d01c77434679a1d04c782d175304"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:20.946660 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_e4c2db87fb024027b4a6e457a8524f10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:20.947219 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e4c2db87fb024027b4a6e457a8524f10 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.948719 28656 tablet_service.cc:1511] Processing CreateTablet for tablet 8be4374ae7cf4907940ae90299bf9ba0 (DEFAULT_TABLE table=default.Table_0a13b90182654346acbbc8f47a56436d [id=c31d65adb55f458c9ced14d598464f89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.949200 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8be4374ae7cf4907940ae90299bf9ba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.949635 28798 consensus_queue.cc:260] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.950255 28798 ts_tablet_manager.cc:1434] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.019s user 0.001s sys 0.001s
I20260618 13:59:20.950604 28798 tablet_bootstrap.cc:492] T f4447a3e87bd476e84f59dd4c4185b26 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:20.952229 28798 tablet_bootstrap.cc:654] T f4447a3e87bd476e84f59dd4c4185b26 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:20.953850 28650 tablet_service.cc:1511] Processing CreateTablet for tablet bab50e1f17a74eee8cad942525002996 (DEFAULT_TABLE table=default.Table_8f22977cfa824da99508b4063a05f8e6 [id=b527fd0afd76403f94cb5fe9f4c95094]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.954269 28650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bab50e1f17a74eee8cad942525002996. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.957834 28529 tablet_service.cc:1511] Processing CreateTablet for tablet 8be4374ae7cf4907940ae90299bf9ba0 (DEFAULT_TABLE table=default.Table_0a13b90182654346acbbc8f47a56436d [id=c31d65adb55f458c9ced14d598464f89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.958328 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8be4374ae7cf4907940ae90299bf9ba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.959309 28661 tablet_service.cc:1511] Processing CreateTablet for tablet 95db5ddaf5d5486988750efed5972eb6 (DEFAULT_TABLE table=default.Table_067483ebd76c49e595543cc25ff1db56 [id=5dadba9b472745259796cf9015e88c65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.959690 28661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95db5ddaf5d5486988750efed5972eb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.959728 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 8c9d89fa75174a969ded07e0596db99b (table default.Table_07cff5be28854452b80fdac31e7ecba3 [id=b024b7c54de64df0a4047d8f0381a5c8]) successfully deleted
I20260618 13:59:20.962800 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 5df99b28e9ae4b6bbad0fe0dafd125d6 (DEFAULT_TABLE table=default.Table_2bf9616ba425458d8c7a4e9a6854d122 [id=e3f4e13202da42a786d3376ba662b31a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.963228 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5df99b28e9ae4b6bbad0fe0dafd125d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.963666 28654 tablet_service.cc:1511] Processing CreateTablet for tablet dca54111fea047d9ae0fabe376d89382 (DEFAULT_TABLE table=default.Table_30c1b9baedbf4ed588651ac692eff7b6 [id=fcccef3123f14001be3f111806a209a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.963758 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 3752f49c9fc64071a45e0ce4ff4cfef2 (DEFAULT_TABLE table=default.Table_4ed45da8b1ed4ff4ad05e46947caf9df [id=af7cd415b54c4e4d8c7fb6f29dc8ff7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.964069 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dca54111fea047d9ae0fabe376d89382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.964293 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3752f49c9fc64071a45e0ce4ff4cfef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.966835 28522 tablet_service.cc:1511] Processing CreateTablet for tablet bab50e1f17a74eee8cad942525002996 (DEFAULT_TABLE table=default.Table_8f22977cfa824da99508b4063a05f8e6 [id=b527fd0afd76403f94cb5fe9f4c95094]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:20.967402 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0827d01c77434679a1d04c782d175304 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:20.971895 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 08a2dee3ac5a4bec96445a6001717eda (DEFAULT_TABLE table=default.Table_d81b926f20864cfe9915d642c53fa6f2 [id=80c0bb8d7847432783cb5c6c0bf76633]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.972323 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08a2dee3ac5a4bec96445a6001717eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.979079 28654 tablet_service.cc:1511] Processing CreateTablet for tablet b0c9d4131ea24809ac081e7a73a21e1c (DEFAULT_TABLE table=default.Table_03e5ab6f90514a49815d0f0c6c8e5f80 [id=0bd492e029e346f68f2db501d798ace1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.979568 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0c9d4131ea24809ac081e7a73a21e1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.982473 28383 tablet_service.cc:1511] Processing CreateTablet for tablet dca54111fea047d9ae0fabe376d89382 (DEFAULT_TABLE table=default.Table_30c1b9baedbf4ed588651ac692eff7b6 [id=fcccef3123f14001be3f111806a209a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:20.986553 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dca54111fea047d9ae0fabe376d89382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:20.991379 28797 tablet_bootstrap.cc:492] T a2ca1e22c38e49d793d413e9eaaac06f P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:20.991551 28797 ts_tablet_manager.cc:1403] T a2ca1e22c38e49d793d413e9eaaac06f P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.080s user 0.009s sys 0.029s
I20260618 13:59:20.992230 28807 raft_consensus.cc:493] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:20.992264 28797 raft_consensus.cc:359] T a2ca1e22c38e49d793d413e9eaaac06f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:20.992455 28797 raft_consensus.cc:385] T a2ca1e22c38e49d793d413e9eaaac06f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:20.992534 28797 raft_consensus.cc:740] T a2ca1e22c38e49d793d413e9eaaac06f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:20.992419 28807 raft_consensus.cc:515] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:20.992760 28797 consensus_queue.cc:260] T a2ca1e22c38e49d793d413e9eaaac06f P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:20.993189 28807 leader_election.cc:290] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:20.996285 28797 ts_tablet_manager.cc:1434] T a2ca1e22c38e49d793d413e9eaaac06f P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260618 13:59:20.997946 28797 tablet_bootstrap.cc:492] T 144aa8984a44410fb3f90dc99e9b2274 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:20.999737 28797 tablet_bootstrap.cc:654] T 144aa8984a44410fb3f90dc99e9b2274 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.003559 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bab50e1f17a74eee8cad942525002996. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.004783 28797 tablet_bootstrap.cc:492] T 144aa8984a44410fb3f90dc99e9b2274 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
W20260618 13:59:21.004732 28053 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 78795.2us
I20260618 13:59:21.005427 28797 ts_tablet_manager.cc:1403] T 144aa8984a44410fb3f90dc99e9b2274 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260618 13:59:21.006263 28797 raft_consensus.cc:359] T 144aa8984a44410fb3f90dc99e9b2274 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.006466 28797 raft_consensus.cc:385] T 144aa8984a44410fb3f90dc99e9b2274 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.006553 28797 raft_consensus.cc:740] T 144aa8984a44410fb3f90dc99e9b2274 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.006808 28797 consensus_queue.cc:260] T 144aa8984a44410fb3f90dc99e9b2274 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.008039 28525 tablet_service.cc:1511] Processing CreateTablet for tablet 95db5ddaf5d5486988750efed5972eb6 (DEFAULT_TABLE table=default.Table_067483ebd76c49e595543cc25ff1db56 [id=5dadba9b472745259796cf9015e88c65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.009321 28797 ts_tablet_manager.cc:1434] T 144aa8984a44410fb3f90dc99e9b2274 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:21.009629 28797 tablet_bootstrap.cc:492] T 102a500268574e428cb2d45e52849f57 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:21.009773 28520 tablet_service.cc:1511] Processing CreateTablet for tablet 08a2dee3ac5a4bec96445a6001717eda (DEFAULT_TABLE table=default.Table_d81b926f20864cfe9915d642c53fa6f2 [id=80c0bb8d7847432783cb5c6c0bf76633]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.012148 28528 tablet_service.cc:1511] Processing CreateTablet for tablet dca54111fea047d9ae0fabe376d89382 (DEFAULT_TABLE table=default.Table_30c1b9baedbf4ed588651ac692eff7b6 [id=fcccef3123f14001be3f111806a209a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.012372 28797 tablet_bootstrap.cc:654] T 102a500268574e428cb2d45e52849f57 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.012718 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95db5ddaf5d5486988750efed5972eb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.016887 28527 tablet_service.cc:1511] Processing CreateTablet for tablet b0c9d4131ea24809ac081e7a73a21e1c (DEFAULT_TABLE table=default.Table_03e5ab6f90514a49815d0f0c6c8e5f80 [id=0bd492e029e346f68f2db501d798ace1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.017316 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0c9d4131ea24809ac081e7a73a21e1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.039852 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "717c38fbd9cb492b9547372e2160f84f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:21.040222 28683 raft_consensus.cc:2468] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:21.040541 28390 tablet_service.cc:1511] Processing CreateTablet for tablet 08a2dee3ac5a4bec96445a6001717eda (DEFAULT_TABLE table=default.Table_d81b926f20864cfe9915d642c53fa6f2 [id=80c0bb8d7847432783cb5c6c0bf76633]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.040875 28385 tablet_service.cc:1511] Processing CreateTablet for tablet b0c9d4131ea24809ac081e7a73a21e1c (DEFAULT_TABLE table=default.Table_03e5ab6f90514a49815d0f0c6c8e5f80 [id=0bd492e029e346f68f2db501d798ace1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.042551 28346 leader_election.cc:304] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:21.042888 28807 raft_consensus.cc:2804] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:21.044198 28524 tablet_service.cc:1511] Processing CreateTablet for tablet 3752f49c9fc64071a45e0ce4ff4cfef2 (DEFAULT_TABLE table=default.Table_4ed45da8b1ed4ff4ad05e46947caf9df [id=af7cd415b54c4e4d8c7fb6f29dc8ff7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.046332 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dca54111fea047d9ae0fabe376d89382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.048220 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "717c38fbd9cb492b9547372e2160f84f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:21.048662 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3752f49c9fc64071a45e0ce4ff4cfef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.051935 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08a2dee3ac5a4bec96445a6001717eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.062170 28388 tablet_service.cc:1511] Processing CreateTablet for tablet 3752f49c9fc64071a45e0ce4ff4cfef2 (DEFAULT_TABLE table=default.Table_4ed45da8b1ed4ff4ad05e46947caf9df [id=af7cd415b54c4e4d8c7fb6f29dc8ff7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.066540 28390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08a2dee3ac5a4bec96445a6001717eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.066828 28807 raft_consensus.cc:493] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:21.067288 28807 raft_consensus.cc:3060] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:21.067255 28346 leader_election.cc:343] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:21.068002 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0c9d4131ea24809ac081e7a73a21e1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.069228 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3752f49c9fc64071a45e0ce4ff4cfef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.070279 28798 tablet_bootstrap.cc:492] T f4447a3e87bd476e84f59dd4c4185b26 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:21.070652 28798 ts_tablet_manager.cc:1403] T f4447a3e87bd476e84f59dd4c4185b26 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.120s user 0.016s sys 0.034s
I20260618 13:59:21.072693 28798 raft_consensus.cc:359] T f4447a3e87bd476e84f59dd4c4185b26 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.076083 28798 raft_consensus.cc:385] T f4447a3e87bd476e84f59dd4c4185b26 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.087827 28807 raft_consensus.cc:515] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.089418 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "717c38fbd9cb492b9547372e2160f84f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:21.089659 28683 raft_consensus.cc:3060] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:21.076234 28798 raft_consensus.cc:740] T f4447a3e87bd476e84f59dd4c4185b26 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.090538 28807 leader_election.cc:290] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:21.090745 28798 consensus_queue.cc:260] T f4447a3e87bd476e84f59dd4c4185b26 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.091442 28798 ts_tablet_manager.cc:1434] T f4447a3e87bd476e84f59dd4c4185b26 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.021s user 0.002s sys 0.000s
I20260618 13:59:21.091843 28798 tablet_bootstrap.cc:492] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:21.094100 28798 tablet_bootstrap.cc:654] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.095984 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "717c38fbd9cb492b9547372e2160f84f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:21.096599 28346 leader_election.cc:343] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:21.098099 28683 raft_consensus.cc:2468] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:21.098956 28346 leader_election.cc:304] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:21.099659 28807 raft_consensus.cc:2804] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:21.101806 28799 tablet_bootstrap.cc:492] T ee783d2c7e274158b3de2cc186f50903 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.101963 28799 ts_tablet_manager.cc:1403] T ee783d2c7e274158b3de2cc186f50903 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.313s user 0.086s sys 0.092s
I20260618 13:59:21.102938 28799 raft_consensus.cc:359] T ee783d2c7e274158b3de2cc186f50903 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.103209 28799 raft_consensus.cc:385] T ee783d2c7e274158b3de2cc186f50903 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.103328 28799 raft_consensus.cc:740] T ee783d2c7e274158b3de2cc186f50903 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.103598 28799 consensus_queue.cc:260] T ee783d2c7e274158b3de2cc186f50903 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.115175 28799 ts_tablet_manager.cc:1434] T ee783d2c7e274158b3de2cc186f50903 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.013s user 0.003s sys 0.007s
I20260618 13:59:21.117286 28799 tablet_bootstrap.cc:492] T b312fbf800434ecb840dd10347efe6e8 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:21.119242 28799 tablet_bootstrap.cc:654] T b312fbf800434ecb840dd10347efe6e8 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:21.131100 28053 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 45830.7us
I20260618 13:59:21.139674 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 5df99b28e9ae4b6bbad0fe0dafd125d6 (DEFAULT_TABLE table=default.Table_2bf9616ba425458d8c7a4e9a6854d122 [id=e3f4e13202da42a786d3376ba662b31a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.140187 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5df99b28e9ae4b6bbad0fe0dafd125d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.149501 28807 raft_consensus.cc:697] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:21.152149 28807 consensus_queue.cc:237] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.153121 28229 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet ebf502f25e9b4342bfaa58f42db7f2f5
I20260618 13:59:21.154557 28229 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 397c7abe26f4451298558d6a0875b677 (table default.Table_2be2c24d66ae4f1d8bf1e78d134841c7 [id=ce780967c12a412a8d7f0bef2f9c00e3]). Expected version 5 got 0
I20260618 13:59:21.155100 28799 tablet_bootstrap.cc:492] T b312fbf800434ecb840dd10347efe6e8 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.155319 28229 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_af38438888374cf2b0b8f3db85810efc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:21.156095 28229 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_af38438888374cf2b0b8f3db85810efc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.158993 28227 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet bd6b0cdffd5a44b5b45b885f6e88eb9d
I20260618 13:59:21.159719 28799 ts_tablet_manager.cc:1403] T b312fbf800434ecb840dd10347efe6e8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.009s
I20260618 13:59:21.159827 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_e8ae99dfd8e84d7bb2723eb5fe129c2f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:21.160326 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e8ae99dfd8e84d7bb2723eb5fe129c2f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.161849 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet ebf502f25e9b4342bfaa58f42db7f2f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.162192 28799 raft_consensus.cc:359] T b312fbf800434ecb840dd10347efe6e8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.162397 28799 raft_consensus.cc:385] T b312fbf800434ecb840dd10347efe6e8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.162515 28799 raft_consensus.cc:740] T b312fbf800434ecb840dd10347efe6e8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.162787 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet bd6b0cdffd5a44b5b45b885f6e88eb9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.162845 28799 consensus_queue.cc:260] T b312fbf800434ecb840dd10347efe6e8 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:21.163175 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet ebf502f25e9b4342bfaa58f42db7f2f5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ebf502f25e9b4342bfaa58f42db7f2f5 already in progress: creating tablet
W20260618 13:59:21.163558 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet bd6b0cdffd5a44b5b45b885f6e88eb9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd6b0cdffd5a44b5b45b885f6e88eb9d already in progress: creating tablet
I20260618 13:59:21.164356 28799 ts_tablet_manager.cc:1434] T b312fbf800434ecb840dd10347efe6e8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260618 13:59:21.165854 28799 tablet_bootstrap.cc:492] T 144aa8984a44410fb3f90dc99e9b2274 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:21.166831 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet ebf502f25e9b4342bfaa58f42db7f2f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.167689 28799 tablet_bootstrap.cc:654] T 144aa8984a44410fb3f90dc99e9b2274 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.167949 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet ebf502f25e9b4342bfaa58f42db7f2f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.169045 28798 tablet_bootstrap.cc:492] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:21.170573 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet bd6b0cdffd5a44b5b45b885f6e88eb9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.173130 28806 raft_consensus.cc:493] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:21.173300 28806 raft_consensus.cc:515] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.173928 28806 leader_election.cc:290] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:21.174530 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "397c7abe26f4451298558d6a0875b677" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:21.174858 28411 raft_consensus.cc:2468] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:21.175288 28617 leader_election.cc:304] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:21.175678 28806 raft_consensus.cc:2804] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:21.175769 28806 raft_consensus.cc:493] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:21.175849 28806 raft_consensus.cc:3060] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:21.179349 28806 raft_consensus.cc:515] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.180440 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "397c7abe26f4451298558d6a0875b677" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:21.180589 28411 raft_consensus.cc:3060] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:21.182641 28798 ts_tablet_manager.cc:1403] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.091s user 0.001s sys 0.009s
I20260618 13:59:21.183420 28411 raft_consensus.cc:2468] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:21.183678 28798 raft_consensus.cc:359] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.183797 28617 leader_election.cc:304] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:21.183877 28798 raft_consensus.cc:385] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.183966 28798 raft_consensus.cc:740] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.170037 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet bd6b0cdffd5a44b5b45b885f6e88eb9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.179083 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "397c7abe26f4451298558d6a0875b677" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:21.184417 28549 raft_consensus.cc:2468] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:21.186002 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "397c7abe26f4451298558d6a0875b677" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:21.186180 28549 raft_consensus.cc:3060] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:21.188972 28549 raft_consensus.cc:2468] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:21.189729 28806 leader_election.cc:290] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:21.190230 28806 raft_consensus.cc:2804] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:21.190573 28806 raft_consensus.cc:697] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:21.190827 28228 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 144aa8984a44410fb3f90dc99e9b2274
I20260618 13:59:21.191267 28806 consensus_queue.cc:237] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.191376 28226 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet b739fa1847184290833e13a2515f3a3a
I20260618 13:59:21.191919 28226 catalog_manager.cc:6784] ReplaceTablet: tablet b739fa1847184290833e13a2515f3a3a deleted and replaced by tablet 36b8a46a41e543c58a946b6764618c7d
I20260618 13:59:21.192193 28226 master_service.cc:960] ReplaceTablet: received request to replace tablet ffd7d5ebaa94416a8eaf3e61bed945e1 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:21.192849 28228 master_service.cc:960] ReplaceTablet: received request to replace tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:21.193634 28225 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 15bbf81e3fbf4927b822afa0635909eb
I20260618 13:59:21.193554 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_4cf88aaddada4361837d7d49170f029b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:21.194172 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4cf88aaddada4361837d7d49170f029b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.194464 28225 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_08ed798477b446ac8960ab3aa1609b8a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:21.195029 28225 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_08ed798477b446ac8960ab3aa1609b8a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.196683 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_1c0c9338a74e4ac8aea2fdeea2495aec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:21.196820 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet b739fa1847184290833e13a2515f3a3a with delete_type TABLET_DATA_DELETED (replaced by tablet 36b8a46a41e543c58a946b6764618c7d) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.197114 28388 tablet_service.cc:1558] Processing DeleteTablet for tablet 15bbf81e3fbf4927b822afa0635909eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:21.197283 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1c0c9338a74e4ac8aea2fdeea2495aec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:21.197741 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet b739fa1847184290833e13a2515f3a3a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b739fa1847184290833e13a2515f3a3a already in progress: creating tablet
W20260618 13:59:21.197993 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 15bbf81e3fbf4927b822afa0635909eb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 15bbf81e3fbf4927b822afa0635909eb already in progress: creating tablet
I20260618 13:59:21.198613 28221 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 5df99b28e9ae4b6bbad0fe0dafd125d6
I20260618 13:59:21.199378 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_d202b6dfbdee4280bdc0bc6e2c7094be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:21.199920 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d202b6dfbdee4280bdc0bc6e2c7094be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.203056 28224 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_4ed45da8b1ed4ff4ad05e46947caf9df" } new_table_name: "default.Table_046616bed79749d5923cbd9ab9dff2b4" modify_external_catalogs: true
I20260618 13:59:21.203337 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 144aa8984a44410fb3f90dc99e9b2274 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.204881 28223 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_4ac6ae42a7f94b79abb170afc27cd978" } new_table_name: "default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9" modify_external_catalogs: true
I20260618 13:59:21.206840 28220 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_edbb2b5290e34792abb3602c8847fff8" } new_table_name: "default.Table_e9f01ff23601409189a5923e3755f4f8" modify_external_catalogs: true
W20260618 13:59:21.208585 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 144aa8984a44410fb3f90dc99e9b2274 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 144aa8984a44410fb3f90dc99e9b2274 already in progress: creating tablet
I20260618 13:59:21.208762 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 5df99b28e9ae4b6bbad0fe0dafd125d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:21.209324 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 5df99b28e9ae4b6bbad0fe0dafd125d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5df99b28e9ae4b6bbad0fe0dafd125d6 already in progress: creating tablet
I20260618 13:59:21.209399 28227 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_58d6a901ec6a4741b232e4d1fe515cb5" } new_table_name: "default.Table_67044988023049af9b42c306e291361c" modify_external_catalogs: true
I20260618 13:59:21.211947 28229 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_1de2d9113caa4693a242a5ec6f6ee0c1" } new_table_name: "default.Table_e9839256914f48e6bb6480c5db7eb379" modify_external_catalogs: true
W20260618 13:59:21.217702 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet ebf502f25e9b4342bfaa58f42db7f2f5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ebf502f25e9b4342bfaa58f42db7f2f5 already in progress: creating tablet
W20260618 13:59:21.218081 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet bd6b0cdffd5a44b5b45b885f6e88eb9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd6b0cdffd5a44b5b45b885f6e88eb9d already in progress: creating tablet
W20260618 13:59:21.218317 28216 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.61:45967, user_credentials={real_user=slave}} blocked reactor thread for 67617.3us
W20260618 13:59:21.223054 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet ebf502f25e9b4342bfaa58f42db7f2f5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ebf502f25e9b4342bfaa58f42db7f2f5 already in progress: creating tablet
W20260618 13:59:21.223429 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet bd6b0cdffd5a44b5b45b885f6e88eb9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd6b0cdffd5a44b5b45b885f6e88eb9d already in progress: creating tablet
I20260618 13:59:21.224574 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 144aa8984a44410fb3f90dc99e9b2274 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.224959 28800 tablet_replica.cc:333] T 144aa8984a44410fb3f90dc99e9b2274 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:21.225189 28800 raft_consensus.cc:2243] T 144aa8984a44410fb3f90dc99e9b2274 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.225348 28800 raft_consensus.cc:2272] T 144aa8984a44410fb3f90dc99e9b2274 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.225812 28798 consensus_queue.cc:260] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.226760 28800 ts_tablet_manager.cc:1916] T 144aa8984a44410fb3f90dc99e9b2274 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.227104 28799 tablet_bootstrap.cc:492] T 144aa8984a44410fb3f90dc99e9b2274 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.227236 28799 ts_tablet_manager.cc:1403] T 144aa8984a44410fb3f90dc99e9b2274 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.062s user 0.003s sys 0.027s
I20260618 13:59:21.228000 28799 raft_consensus.cc:359] T 144aa8984a44410fb3f90dc99e9b2274 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.228176 28799 raft_consensus.cc:385] T 144aa8984a44410fb3f90dc99e9b2274 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.228253 28799 raft_consensus.cc:740] T 144aa8984a44410fb3f90dc99e9b2274 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.228467 28799 consensus_queue.cc:260] T 144aa8984a44410fb3f90dc99e9b2274 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.232753 28800 ts_tablet_manager.cc:1929] T 144aa8984a44410fb3f90dc99e9b2274 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260618 13:59:21.233000 28216 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet eeffc96b9b97441d9301e4919d0047e1 (table default.Table_f0607bf6529e4f209e00f49908420141 [id=972bc2c124ac459da06ec4093c5b564b], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet eeffc96b9b97441d9301e4919d0047e1 consensus state has no leader
I20260618 13:59:21.233047 28799 ts_tablet_manager.cc:1434] T 144aa8984a44410fb3f90dc99e9b2274 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260618 13:59:21.233321 28800 log.cc:1199] T 144aa8984a44410fb3f90dc99e9b2274 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/144aa8984a44410fb3f90dc99e9b2274
I20260618 13:59:21.233749 28800 ts_tablet_manager.cc:1950] T 144aa8984a44410fb3f90dc99e9b2274 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:21.234866 28799 tablet_bootstrap.cc:492] T a2ca1e22c38e49d793d413e9eaaac06f P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:21.237749 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet b739fa1847184290833e13a2515f3a3a with delete_type TABLET_DATA_DELETED (replaced by tablet 36b8a46a41e543c58a946b6764618c7d) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.235342 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet 144aa8984a44410fb3f90dc99e9b2274 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.237331 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet b739fa1847184290833e13a2515f3a3a with delete_type TABLET_DATA_DELETED (replaced by tablet 36b8a46a41e543c58a946b6764618c7d) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.240219 28799 tablet_bootstrap.cc:654] T a2ca1e22c38e49d793d413e9eaaac06f P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.242864 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet f4447a3e87bd476e84f59dd4c4185b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.243131 28801 tablet_replica.cc:333] T f4447a3e87bd476e84f59dd4c4185b26 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:21.243337 28801 raft_consensus.cc:2243] T f4447a3e87bd476e84f59dd4c4185b26 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.243522 28801 raft_consensus.cc:2272] T f4447a3e87bd476e84f59dd4c4185b26 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.244230 28801 ts_tablet_manager.cc:1916] T f4447a3e87bd476e84f59dd4c4185b26 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.246618 28798 ts_tablet_manager.cc:1434] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.064s user 0.001s sys 0.001s
I20260618 13:59:21.247049 28798 tablet_bootstrap.cc:492] T 91c250dfbd72416d9d74f6cbb67b2bdf P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:21.248870 28798 tablet_bootstrap.cc:654] T 91c250dfbd72416d9d74f6cbb67b2bdf P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.250423 28801 ts_tablet_manager.cc:1929] T f4447a3e87bd476e84f59dd4c4185b26 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.250494 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 15bbf81e3fbf4927b822afa0635909eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.250609 28801 log.cc:1199] T f4447a3e87bd476e84f59dd4c4185b26 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/f4447a3e87bd476e84f59dd4c4185b26
I20260618 13:59:21.250895 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet 15bbf81e3fbf4927b822afa0635909eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.250995 28801 ts_tablet_manager.cc:1950] T f4447a3e87bd476e84f59dd4c4185b26 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
W20260618 13:59:21.253976 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 144aa8984a44410fb3f90dc99e9b2274 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 144aa8984a44410fb3f90dc99e9b2274 already in progress: creating tablet
W20260618 13:59:21.254212 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet b739fa1847184290833e13a2515f3a3a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b739fa1847184290833e13a2515f3a3a already in progress: creating tablet
I20260618 13:59:21.254382 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet f4447a3e87bd476e84f59dd4c4185b26 (table default.Table_4cf6e1e03c724f42a0d15aed65142720 [id=80246ec4fbca44248528c2c379034cee]) successfully deleted
I20260618 13:59:21.255354 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet 5df99b28e9ae4b6bbad0fe0dafd125d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.255476 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 144aa8984a44410fb3f90dc99e9b2274 (table default.Table_56009fb08789451f82400e86054dd066 [id=c2109218729e47e0b1057c9802e312a4]) successfully deleted
I20260618 13:59:21.255609 28797 tablet_bootstrap.cc:492] T 102a500268574e428cb2d45e52849f57 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
W20260618 13:59:21.255678 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet b739fa1847184290833e13a2515f3a3a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b739fa1847184290833e13a2515f3a3a already in progress: creating tablet
I20260618 13:59:21.255777 28797 ts_tablet_manager.cc:1403] T 102a500268574e428cb2d45e52849f57 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.246s user 0.027s sys 0.088s
W20260618 13:59:21.255873 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 15bbf81e3fbf4927b822afa0635909eb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 15bbf81e3fbf4927b822afa0635909eb already in progress: creating tablet
W20260618 13:59:21.256090 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 15bbf81e3fbf4927b822afa0635909eb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 15bbf81e3fbf4927b822afa0635909eb already in progress: creating tablet
I20260618 13:59:21.256611 28797 raft_consensus.cc:359] T 102a500268574e428cb2d45e52849f57 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.256850 28797 raft_consensus.cc:385] T 102a500268574e428cb2d45e52849f57 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.256930 28797 raft_consensus.cc:740] T 102a500268574e428cb2d45e52849f57 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.257212 28797 consensus_queue.cc:260] T 102a500268574e428cb2d45e52849f57 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.258611 28806 raft_consensus.cc:493] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:21.258759 28806 raft_consensus.cc:515] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.258874 28797 ts_tablet_manager.cc:1434] T 102a500268574e428cb2d45e52849f57 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:21.259479 28806 leader_election.cc:290] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:21.259711 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 5df99b28e9ae4b6bbad0fe0dafd125d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.260089 28797 tablet_bootstrap.cc:492] T b312fbf800434ecb840dd10347efe6e8 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:21.260088 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "570d36a27b7d417989ad72885d38ff63" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:21.260870 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "570d36a27b7d417989ad72885d38ff63" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:21.261631 28617 leader_election.cc:343] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:21.262108 28797 tablet_bootstrap.cc:654] T b312fbf800434ecb840dd10347efe6e8 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:21.262499 28618 leader_election.cc:343] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:21.262646 28618 leader_election.cc:304] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:21.263180 28806 raft_consensus.cc:2749] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:21.263702 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 5df99b28e9ae4b6bbad0fe0dafd125d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5df99b28e9ae4b6bbad0fe0dafd125d6 already in progress: creating tablet
W20260618 13:59:21.264030 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 5df99b28e9ae4b6bbad0fe0dafd125d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5df99b28e9ae4b6bbad0fe0dafd125d6 already in progress: creating tablet
I20260618 13:59:21.264376 28798 tablet_bootstrap.cc:492] T 91c250dfbd72416d9d74f6cbb67b2bdf P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:21.264536 28798 ts_tablet_manager.cc:1403] T 91c250dfbd72416d9d74f6cbb67b2bdf P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.000s
I20260618 13:59:21.266884 28799 tablet_bootstrap.cc:492] T a2ca1e22c38e49d793d413e9eaaac06f P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.267130 28799 ts_tablet_manager.cc:1403] T a2ca1e22c38e49d793d413e9eaaac06f P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.003s
I20260618 13:59:21.268682 28799 raft_consensus.cc:359] T a2ca1e22c38e49d793d413e9eaaac06f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.268994 28799 raft_consensus.cc:385] T a2ca1e22c38e49d793d413e9eaaac06f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.269127 28799 raft_consensus.cc:740] T a2ca1e22c38e49d793d413e9eaaac06f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.269498 28799 consensus_queue.cc:260] T a2ca1e22c38e49d793d413e9eaaac06f P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.270399 28799 ts_tablet_manager.cc:1434] T a2ca1e22c38e49d793d413e9eaaac06f P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260618 13:59:21.270740 28226 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet ffd7d5ebaa94416a8eaf3e61bed945e1
I20260618 13:59:21.271225 28226 catalog_manager.cc:6784] ReplaceTablet: tablet ffd7d5ebaa94416a8eaf3e61bed945e1 deleted and replaced by tablet d6871232860b41d489e34d95eb897241
I20260618 13:59:21.272444 28799 tablet_bootstrap.cc:492] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:21.274712 28225 master_service.cc:960] ReplaceTablet: received request to replace tablet b8389b77e6b8407db38e07e85f7509b1 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:21.274861 28799 tablet_bootstrap.cc:654] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.275254 28222 master_service.cc:960] ReplaceTablet: received request to replace tablet 717c38fbd9cb492b9547372e2160f84f from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:21.279161 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.279423 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet ffd7d5ebaa94416a8eaf3e61bed945e1 with delete_type TABLET_DATA_DELETED (replaced by tablet d6871232860b41d489e34d95eb897241) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.279762 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet ffd7d5ebaa94416a8eaf3e61bed945e1 with delete_type TABLET_DATA_DELETED (replaced by tablet d6871232860b41d489e34d95eb897241) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.280999 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet a2ca1e22c38e49d793d413e9eaaac06f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
W20260618 13:59:21.281924 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet ffd7d5ebaa94416a8eaf3e61bed945e1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffd7d5ebaa94416a8eaf3e61bed945e1 already in progress: creating tablet
I20260618 13:59:21.282900 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet ffd7d5ebaa94416a8eaf3e61bed945e1 with delete_type TABLET_DATA_DELETED (replaced by tablet d6871232860b41d489e34d95eb897241) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.283154 28800 tablet_replica.cc:333] T a2ca1e22c38e49d793d413e9eaaac06f P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:21.283493 28800 raft_consensus.cc:2243] T a2ca1e22c38e49d793d413e9eaaac06f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260618 13:59:21.289096 28224 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3752f49c9fc64071a45e0ce4ff4cfef2 consensus state has no leader
I20260618 13:59:21.289254 28800 raft_consensus.cc:2272] T a2ca1e22c38e49d793d413e9eaaac06f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:21.289589 28223 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4ebed81a21b14db48ecb69299e0956cd consensus state has no leader
W20260618 13:59:21.289958 28220 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 37fedd27109944e2b3c870418c117758 consensus state has no leader
W20260618 13:59:21.290380 28227 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 29bf263d20934ed6a6efb4b215252c00 consensus state has no leader
I20260618 13:59:21.290558 28800 ts_tablet_manager.cc:1916] T a2ca1e22c38e49d793d413e9eaaac06f P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.290513 28801 tablet_replica.cc:333] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:21.290973 28801 raft_consensus.cc:2243] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.291224 28801 raft_consensus.cc:2272] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:21.293540 28229 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ba0d05ea44e1473fb4b24f61eab096e4 consensus state has no leader
I20260618 13:59:21.294965 28221 catalog_manager.cc:5696] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:21.295944 28221 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 397c7abe26f4451298558d6a0875b677 (table default.Table_2be2c24d66ae4f1d8bf1e78d134841c7 [id=ce780967c12a412a8d7f0bef2f9c00e3]). Expected version 5 got 0
I20260618 13:59:21.296797 28389 tablet_service.cc:1511] Processing CreateTablet for tablet e025b64380314546af8a1ab90c817825 (DEFAULT_TABLE table=default.Table_c940d6e3d5d34adfaf39c7b5b60044e1 [id=803d5220ce1a4dcfaf27d93c574be2ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.297231 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e025b64380314546af8a1ab90c817825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.299162 28800 ts_tablet_manager.cc:1929] T a2ca1e22c38e49d793d413e9eaaac06f P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.299321 28800 log.cc:1199] T a2ca1e22c38e49d793d413e9eaaac06f P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/a2ca1e22c38e49d793d413e9eaaac06f
I20260618 13:59:21.299469 28388 tablet_service.cc:1511] Processing CreateTablet for tablet 68b8293a7c924a449307ab88a1cf9290 (DEFAULT_TABLE table=default.Table_c97dcabd4c2b45a18f445a099ec7ace9 [id=38bde7a301814aa285e4bd30bcc527aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.299453 28226 catalog_manager.cc:5696] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:21.299780 28800 ts_tablet_manager.cc:1950] T a2ca1e22c38e49d793d413e9eaaac06f P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:21.299827 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68b8293a7c924a449307ab88a1cf9290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.299927 28226 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 717c38fbd9cb492b9547372e2160f84f (table default.Table_a2f92d4bf5bc49aca105a2886053d230 [id=3bcc877eddec4d00b3dc1760cc635b36]). Expected version 5 got 0
I20260618 13:59:21.301162 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet a2ca1e22c38e49d793d413e9eaaac06f (table default.Table_5224a46184b74b75a0379955524146f3 [id=f071d496d807405cb0f3b022757b85e0]) successfully deleted
I20260618 13:59:21.301710 28655 tablet_service.cc:1511] Processing CreateTablet for tablet e025b64380314546af8a1ab90c817825 (DEFAULT_TABLE table=default.Table_c940d6e3d5d34adfaf39c7b5b60044e1 [id=803d5220ce1a4dcfaf27d93c574be2ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.301961 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 68b8293a7c924a449307ab88a1cf9290 (DEFAULT_TABLE table=default.Table_c97dcabd4c2b45a18f445a099ec7ace9 [id=38bde7a301814aa285e4bd30bcc527aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.302198 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e025b64380314546af8a1ab90c817825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.302317 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68b8293a7c924a449307ab88a1cf9290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.302711 28656 tablet_service.cc:1511] Processing CreateTablet for tablet 68b8293a7c924a449307ab88a1cf9290 (DEFAULT_TABLE table=default.Table_c97dcabd4c2b45a18f445a099ec7ace9 [id=38bde7a301814aa285e4bd30bcc527aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.303076 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68b8293a7c924a449307ab88a1cf9290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.306733 28520 tablet_service.cc:1511] Processing CreateTablet for tablet e025b64380314546af8a1ab90c817825 (DEFAULT_TABLE table=default.Table_c940d6e3d5d34adfaf39c7b5b60044e1 [id=803d5220ce1a4dcfaf27d93c574be2ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.307119 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e025b64380314546af8a1ab90c817825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.320420 28801 ts_tablet_manager.cc:1916] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.320612 28798 raft_consensus.cc:359] T 91c250dfbd72416d9d74f6cbb67b2bdf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.321408 28798 raft_consensus.cc:385] T 91c250dfbd72416d9d74f6cbb67b2bdf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.321547 28798 raft_consensus.cc:740] T 91c250dfbd72416d9d74f6cbb67b2bdf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
W20260618 13:59:21.322881 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet ffd7d5ebaa94416a8eaf3e61bed945e1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffd7d5ebaa94416a8eaf3e61bed945e1 already in progress: creating tablet
I20260618 13:59:21.323302 28798 consensus_queue.cc:260] T 91c250dfbd72416d9d74f6cbb67b2bdf P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.328132 28225 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet b8389b77e6b8407db38e07e85f7509b1
I20260618 13:59:21.328651 28225 catalog_manager.cc:6784] ReplaceTablet: tablet b8389b77e6b8407db38e07e85f7509b1 deleted and replaced by tablet d9c00003300246ada2a356e3e1cf684b
I20260618 13:59:21.330660 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet b8389b77e6b8407db38e07e85f7509b1 with delete_type TABLET_DATA_DELETED (replaced by tablet d9c00003300246ada2a356e3e1cf684b) from {username='slave'} at 127.0.0.1:43976
W20260618 13:59:21.331432 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet b8389b77e6b8407db38e07e85f7509b1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b8389b77e6b8407db38e07e85f7509b1 already in progress: creating tablet
I20260618 13:59:21.334563 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet a2ca1e22c38e49d793d413e9eaaac06f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.335304 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 102a500268574e428cb2d45e52849f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
W20260618 13:59:21.335592 28215 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet a2ca1e22c38e49d793d413e9eaaac06f because the tablet was not found. No further retry: Not found: Tablet not found: a2ca1e22c38e49d793d413e9eaaac06f
I20260618 13:59:21.336150 28800 tablet_replica.cc:333] T 102a500268574e428cb2d45e52849f57 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:21.336426 28800 raft_consensus.cc:2243] T 102a500268574e428cb2d45e52849f57 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.336648 28800 raft_consensus.cc:2272] T 102a500268574e428cb2d45e52849f57 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.337008 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet a2ca1e22c38e49d793d413e9eaaac06f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.337328 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 144aa8984a44410fb3f90dc99e9b2274 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.337404 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet b312fbf800434ecb840dd10347efe6e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.337589 28802 tablet_replica.cc:333] T a2ca1e22c38e49d793d413e9eaaac06f P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:21.337855 28802 raft_consensus.cc:2243] T a2ca1e22c38e49d793d413e9eaaac06f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.338058 28802 raft_consensus.cc:2272] T a2ca1e22c38e49d793d413e9eaaac06f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.338562 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet b8389b77e6b8407db38e07e85f7509b1 with delete_type TABLET_DATA_DELETED (replaced by tablet d9c00003300246ada2a356e3e1cf684b) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.338812 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet 91c250dfbd72416d9d74f6cbb67b2bdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.338888 28800 ts_tablet_manager.cc:1916] T 102a500268574e428cb2d45e52849f57 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.338912 28802 ts_tablet_manager.cc:1916] T a2ca1e22c38e49d793d413e9eaaac06f P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.339067 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet ee783d2c7e274158b3de2cc186f50903 with delete_type TABLET_DATA_DELETED (replaced by tablet 311ba4dd83984f379c8f4f8274bc26c1) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.339911 28798 ts_tablet_manager.cc:1434] T 91c250dfbd72416d9d74f6cbb67b2bdf P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.075s user 0.001s sys 0.001s
I20260618 13:59:21.339986 28222 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 717c38fbd9cb492b9547372e2160f84f
I20260618 13:59:21.340422 28222 catalog_manager.cc:6784] ReplaceTablet: tablet 717c38fbd9cb492b9547372e2160f84f deleted and replaced by tablet 8b59255b14df4158afe37cb8890e7415
I20260618 13:59:21.341100 28798 tablet_bootstrap.cc:492] T 587b82a6da8d4fe48b26070803831d9d P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:21.342918 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet 717c38fbd9cb492b9547372e2160f84f with delete_type TABLET_DATA_DELETED (replaced by tablet 8b59255b14df4158afe37cb8890e7415) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.344230 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 717c38fbd9cb492b9547372e2160f84f with delete_type TABLET_DATA_DELETED (replaced by tablet 8b59255b14df4158afe37cb8890e7415) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.346159 28388 tablet_service.cc:1558] Processing DeleteTablet for tablet b8389b77e6b8407db38e07e85f7509b1 with delete_type TABLET_DATA_DELETED (replaced by tablet d9c00003300246ada2a356e3e1cf684b) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.346226 28800 ts_tablet_manager.cc:1929] T 102a500268574e428cb2d45e52849f57 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.346411 28800 log.cc:1199] T 102a500268574e428cb2d45e52849f57 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/102a500268574e428cb2d45e52849f57
I20260618 13:59:21.346582 28802 ts_tablet_manager.cc:1929] T a2ca1e22c38e49d793d413e9eaaac06f P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.346740 28802 log.cc:1199] T a2ca1e22c38e49d793d413e9eaaac06f P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/a2ca1e22c38e49d793d413e9eaaac06f
I20260618 13:59:21.346895 28800 ts_tablet_manager.cc:1950] T 102a500268574e428cb2d45e52849f57 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:21.347127 28802 ts_tablet_manager.cc:1950] T a2ca1e22c38e49d793d413e9eaaac06f P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:21.347872 28801 ts_tablet_manager.cc:1929] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.347882 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 102a500268574e428cb2d45e52849f57 (table default.Table_bd146d3bdafe4c66b00a25d98887f1e2 [id=76865706d6a54ec198dd205b6ad61920]) successfully deleted
I20260618 13:59:21.348006 28801 log.cc:1199] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/9fd3fbcf0d9745a1b3d3408f95022bf6
I20260618 13:59:21.347998 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet a2ca1e22c38e49d793d413e9eaaac06f (table default.Table_5224a46184b74b75a0379955524146f3 [id=f071d496d807405cb0f3b022757b85e0]) successfully deleted
I20260618 13:59:21.348201 28800 tablet_replica.cc:333] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:21.348264 28802 tablet_replica.cc:333] T 144aa8984a44410fb3f90dc99e9b2274 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:21.348377 28800 raft_consensus.cc:2243] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.348451 28798 tablet_bootstrap.cc:654] T 587b82a6da8d4fe48b26070803831d9d P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.348444 28802 raft_consensus.cc:2243] T 144aa8984a44410fb3f90dc99e9b2274 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.348572 28800 raft_consensus.cc:2272] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.348691 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 717c38fbd9cb492b9547372e2160f84f with delete_type TABLET_DATA_DELETED (replaced by tablet 8b59255b14df4158afe37cb8890e7415) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.349208 28800 ts_tablet_manager.cc:1916] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.350076 28797 tablet_bootstrap.cc:492] T b312fbf800434ecb840dd10347efe6e8 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:21.350260 28797 ts_tablet_manager.cc:1403] T b312fbf800434ecb840dd10347efe6e8 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.091s user 0.000s sys 0.003s
I20260618 13:59:21.351159 28797 raft_consensus.cc:359] T b312fbf800434ecb840dd10347efe6e8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.351459 28797 raft_consensus.cc:385] T b312fbf800434ecb840dd10347efe6e8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.351562 28797 raft_consensus.cc:740] T b312fbf800434ecb840dd10347efe6e8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.351799 28797 consensus_queue.cc:260] T b312fbf800434ecb840dd10347efe6e8 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.354339 28802 raft_consensus.cc:2272] T 144aa8984a44410fb3f90dc99e9b2274 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.355185 28802 ts_tablet_manager.cc:1916] T 144aa8984a44410fb3f90dc99e9b2274 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.356809 28797 ts_tablet_manager.cc:1434] T b312fbf800434ecb840dd10347efe6e8 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260618 13:59:21.356906 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet b312fbf800434ecb840dd10347efe6e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.358611 28797 tablet_bootstrap.cc:492] T d5cb3137659a4481840fe04174ef682d P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:21.362314 28802 ts_tablet_manager.cc:1929] T 144aa8984a44410fb3f90dc99e9b2274 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.362453 28802 log.cc:1199] T 144aa8984a44410fb3f90dc99e9b2274 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/144aa8984a44410fb3f90dc99e9b2274
I20260618 13:59:21.362684 28520 tablet_service.cc:1511] Processing CreateTablet for tablet fca76774f119435899188313a33dad98 (DEFAULT_TABLE table=default.Table_4cf88aaddada4361837d7d49170f029b [id=50525d6257fa4de8a00dbee44d37f24d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.362838 28802 ts_tablet_manager.cc:1950] T 144aa8984a44410fb3f90dc99e9b2274 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:21.363135 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fca76774f119435899188313a33dad98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.363338 28655 tablet_service.cc:1511] Processing CreateTablet for tablet fca76774f119435899188313a33dad98 (DEFAULT_TABLE table=default.Table_4cf88aaddada4361837d7d49170f029b [id=50525d6257fa4de8a00dbee44d37f24d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.363682 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fca76774f119435899188313a33dad98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.367481 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 144aa8984a44410fb3f90dc99e9b2274 (table default.Table_56009fb08789451f82400e86054dd066 [id=c2109218729e47e0b1057c9802e312a4]) successfully deleted
I20260618 13:59:21.368256 28802 tablet_replica.cc:333] T b312fbf800434ecb840dd10347efe6e8 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:21.368462 28802 raft_consensus.cc:2243] T b312fbf800434ecb840dd10347efe6e8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.368635 28802 raft_consensus.cc:2272] T b312fbf800434ecb840dd10347efe6e8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.369931 28802 ts_tablet_manager.cc:1916] T b312fbf800434ecb840dd10347efe6e8 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.370554 28222 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_823368bbdf8242739a6022f65280f5a2" } modify_external_catalogs: true
I20260618 13:59:21.370651 28797 tablet_bootstrap.cc:654] T d5cb3137659a4481840fe04174ef682d P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.370743 28222 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_823368bbdf8242739a6022f65280f5a2" } modify_external_catalogs: true
I20260618 13:59:21.372334 28520 tablet_service.cc:1511] Processing CreateTablet for tablet a1a06078974e4bfbbc4b949802a39bf1 (DEFAULT_TABLE table=default.Table_d202b6dfbdee4280bdc0bc6e2c7094be [id=7b747ef3b2494ca78c499c5bb75648d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.372789 28226 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_067483ebd76c49e595543cc25ff1db56" } modify_external_catalogs: true
I20260618 13:59:21.372944 28226 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_067483ebd76c49e595543cc25ff1db56" } modify_external_catalogs: true
I20260618 13:59:21.373672 28800 ts_tablet_manager.cc:1929] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.373647 28389 tablet_service.cc:1511] Processing CreateTablet for tablet fca76774f119435899188313a33dad98 (DEFAULT_TABLE table=default.Table_4cf88aaddada4361837d7d49170f029b [id=50525d6257fa4de8a00dbee44d37f24d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.373778 28800 log.cc:1199] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/717c38fbd9cb492b9547372e2160f84f
I20260618 13:59:21.374025 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fca76774f119435899188313a33dad98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.374104 28800 ts_tablet_manager.cc:1950] T 717c38fbd9cb492b9547372e2160f84f P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:21.374742 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1a06078974e4bfbbc4b949802a39bf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.375425 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 717c38fbd9cb492b9547372e2160f84f (table default.Table_a2f92d4bf5bc49aca105a2886053d230 [id=3bcc877eddec4d00b3dc1760cc635b36]) successfully deleted
I20260618 13:59:21.375763 28800 tablet_replica.cc:333] T b312fbf800434ecb840dd10347efe6e8 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:21.375970 28800 raft_consensus.cc:2243] T b312fbf800434ecb840dd10347efe6e8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.375967 28798 tablet_bootstrap.cc:492] T 587b82a6da8d4fe48b26070803831d9d P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:21.376144 28800 raft_consensus.cc:2272] T b312fbf800434ecb840dd10347efe6e8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.376863 28800 ts_tablet_manager.cc:1916] T b312fbf800434ecb840dd10347efe6e8 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.377014 28798 ts_tablet_manager.cc:1403] T 587b82a6da8d4fe48b26070803831d9d P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.036s user 0.004s sys 0.000s
I20260618 13:59:21.377822 28798 raft_consensus.cc:359] T 587b82a6da8d4fe48b26070803831d9d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.378400 28802 ts_tablet_manager.cc:1929] T b312fbf800434ecb840dd10347efe6e8 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.378557 28802 log.cc:1199] T b312fbf800434ecb840dd10347efe6e8 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/b312fbf800434ecb840dd10347efe6e8
I20260618 13:59:21.378556 28798 raft_consensus.cc:385] T 587b82a6da8d4fe48b26070803831d9d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.378751 28798 raft_consensus.cc:740] T 587b82a6da8d4fe48b26070803831d9d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.378904 28802 ts_tablet_manager.cc:1950] T b312fbf800434ecb840dd10347efe6e8 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:21.379869 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet b312fbf800434ecb840dd10347efe6e8 (table default.Table_1ccdff414ebc4079933d0edf079a2fdb [id=7e70d6cbf42e437da0c1bb47eee6633c]) successfully deleted
I20260618 13:59:21.380134 28798 consensus_queue.cc:260] T 587b82a6da8d4fe48b26070803831d9d P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.380837 28801 ts_tablet_manager.cc:1950] T 9fd3fbcf0d9745a1b3d3408f95022bf6 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:21.382622 28798 ts_tablet_manager.cc:1434] T 587b82a6da8d4fe48b26070803831d9d P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260618 13:59:21.385425 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 9fd3fbcf0d9745a1b3d3408f95022bf6 (table default.Table_e10ecf3a190148d0b17f97204f155353 [id=179292b8d0574f7bb5b6edb5dcad5cb9]) successfully deleted
I20260618 13:59:21.385540 28798 tablet_bootstrap.cc:492] T 717c38fbd9cb492b9547372e2160f84f P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:21.386349 28802 tablet_replica.cc:333] T ee783d2c7e274158b3de2cc186f50903 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:21.386566 28802 raft_consensus.cc:2243] T ee783d2c7e274158b3de2cc186f50903 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.386765 28802 raft_consensus.cc:2272] T ee783d2c7e274158b3de2cc186f50903 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.386806 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet 587b82a6da8d4fe48b26070803831d9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:21.387476 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet ffd7d5ebaa94416a8eaf3e61bed945e1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffd7d5ebaa94416a8eaf3e61bed945e1 already in progress: creating tablet
W20260618 13:59:21.387811 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet b8389b77e6b8407db38e07e85f7509b1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b8389b77e6b8407db38e07e85f7509b1 already in progress: creating tablet
I20260618 13:59:21.388057 28801 tablet_replica.cc:333] T 91c250dfbd72416d9d74f6cbb67b2bdf P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:21.388090 28802 ts_tablet_manager.cc:1916] T ee783d2c7e274158b3de2cc186f50903 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.389768 28800 ts_tablet_manager.cc:1929] T b312fbf800434ecb840dd10347efe6e8 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.389905 28800 log.cc:1199] T b312fbf800434ecb840dd10347efe6e8 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/b312fbf800434ecb840dd10347efe6e8
I20260618 13:59:21.389988 28222 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 31b328882af549a78535cdd55c6ea09c
I20260618 13:59:21.390237 28800 ts_tablet_manager.cc:1950] T b312fbf800434ecb840dd10347efe6e8 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:21.390394 28801 raft_consensus.cc:2243] T 91c250dfbd72416d9d74f6cbb67b2bdf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.390861 28388 tablet_service.cc:1558] Processing DeleteTablet for tablet 31b328882af549a78535cdd55c6ea09c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.391587 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 31b328882af549a78535cdd55c6ea09c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.391244 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet 31b328882af549a78535cdd55c6ea09c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.393826 28801 raft_consensus.cc:2272] T 91c250dfbd72416d9d74f6cbb67b2bdf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.394400 28798 tablet_bootstrap.cc:654] T 717c38fbd9cb492b9547372e2160f84f P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.395277 28226 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 95db5ddaf5d5486988750efed5972eb6
I20260618 13:59:21.396358 28388 tablet_service.cc:1558] Processing DeleteTablet for tablet 95db5ddaf5d5486988750efed5972eb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.396966 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet 95db5ddaf5d5486988750efed5972eb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.397562 28802 ts_tablet_manager.cc:1929] T ee783d2c7e274158b3de2cc186f50903 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.397677 28802 log.cc:1199] T ee783d2c7e274158b3de2cc186f50903 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/ee783d2c7e274158b3de2cc186f50903
I20260618 13:59:21.398025 28802 ts_tablet_manager.cc:1950] T ee783d2c7e274158b3de2cc186f50903 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:21.398195 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 95db5ddaf5d5486988750efed5972eb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.397415 28797 tablet_bootstrap.cc:492] T d5cb3137659a4481840fe04174ef682d P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:21.399031 28797 ts_tablet_manager.cc:1403] T d5cb3137659a4481840fe04174ef682d P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.040s user 0.001s sys 0.002s
I20260618 13:59:21.399015 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet ee783d2c7e274158b3de2cc186f50903 (table default.Table_dd833764ce1041cba71575be46d771a9 [id=d8ef984c664f4a6598c70a48e9aff826]) successfully deleted
I20260618 13:59:21.400702 28799 tablet_bootstrap.cc:492] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.400843 28799 ts_tablet_manager.cc:1403] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.130s user 0.032s sys 0.052s
I20260618 13:59:21.401129 28526 tablet_service.cc:1511] Processing CreateTablet for tablet eead369de58f40378254697cbbbc6958 (DEFAULT_TABLE table=default.Table_e8ae99dfd8e84d7bb2723eb5fe129c2f [id=6d8f3597814241f3bc8c920d3a59008e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.401508 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eead369de58f40378254697cbbbc6958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.401765 28388 tablet_service.cc:1511] Processing CreateTablet for tablet eead369de58f40378254697cbbbc6958 (DEFAULT_TABLE table=default.Table_e8ae99dfd8e84d7bb2723eb5fe129c2f [id=6d8f3597814241f3bc8c920d3a59008e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.402171 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eead369de58f40378254697cbbbc6958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.407275 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet b312fbf800434ecb840dd10347efe6e8 (table default.Table_1ccdff414ebc4079933d0edf079a2fdb [id=7e70d6cbf42e437da0c1bb47eee6633c]) successfully deleted
W20260618 13:59:21.407534 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 31b328882af549a78535cdd55c6ea09c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 31b328882af549a78535cdd55c6ea09c already in progress: creating tablet
W20260618 13:59:21.407759 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 95db5ddaf5d5486988750efed5972eb6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 95db5ddaf5d5486988750efed5972eb6 already in progress: creating tablet
I20260618 13:59:21.407737 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 4a01dfb4d59549399c94640f16045555 (DEFAULT_TABLE table=default.Table_7700ee6bae434a01b621ad02088cc5f0 [id=fe7eabed8da6400bbb721520c6182f8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.407920 28391 tablet_service.cc:1511] Processing CreateTablet for tablet 4a01dfb4d59549399c94640f16045555 (DEFAULT_TABLE table=default.Table_7700ee6bae434a01b621ad02088cc5f0 [id=fe7eabed8da6400bbb721520c6182f8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.408095 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a01dfb4d59549399c94640f16045555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.408269 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a01dfb4d59549399c94640f16045555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.408385 28656 tablet_service.cc:1511] Processing CreateTablet for tablet eead369de58f40378254697cbbbc6958 (DEFAULT_TABLE table=default.Table_e8ae99dfd8e84d7bb2723eb5fe129c2f [id=6d8f3597814241f3bc8c920d3a59008e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.408727 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eead369de58f40378254697cbbbc6958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.410666 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 94365c44b5e24acab5df60c67203794a (DEFAULT_TABLE table=default.Table_e4c2db87fb024027b4a6e457a8524f10 [id=770fade529a64ec29e70968dd6d6bb93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.410682 28650 tablet_service.cc:1511] Processing CreateTablet for tablet 864f2602840c40fc8606b3784e671cba (DEFAULT_TABLE table=default.Table_0827d01c77434679a1d04c782d175304 [id=3ac9a6b120d44ac2bae74d85d7bc6930]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.410987 28650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 864f2602840c40fc8606b3784e671cba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.411023 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94365c44b5e24acab5df60c67203794a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.411754 28522 tablet_service.cc:1511] Processing CreateTablet for tablet 4a01dfb4d59549399c94640f16045555 (DEFAULT_TABLE table=default.Table_7700ee6bae434a01b621ad02088cc5f0 [id=fe7eabed8da6400bbb721520c6182f8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.412139 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a01dfb4d59549399c94640f16045555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.412696 28651 tablet_service.cc:1511] Processing CreateTablet for tablet 0a51e830e3014726b1d1c2608973f036 (DEFAULT_TABLE table=default.Table_696ead97d50747228517c265538b25fb [id=716ae493050c4288b9bf041c94523b05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.413002 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a51e830e3014726b1d1c2608973f036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.414695 28385 tablet_service.cc:1511] Processing CreateTablet for tablet edc72a06ee8e4420b07a17d2feb3fcc2 (DEFAULT_TABLE table=default.Table_08ed798477b446ac8960ab3aa1609b8a [id=9d63ad7327a44124af817b465a9ae0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.414739 28528 tablet_service.cc:1511] Processing CreateTablet for tablet 864f2602840c40fc8606b3784e671cba (DEFAULT_TABLE table=default.Table_0827d01c77434679a1d04c782d175304 [id=3ac9a6b120d44ac2bae74d85d7bc6930]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.414912 28529 tablet_service.cc:1511] Processing CreateTablet for tablet 94365c44b5e24acab5df60c67203794a (DEFAULT_TABLE table=default.Table_e4c2db87fb024027b4a6e457a8524f10 [id=770fade529a64ec29e70968dd6d6bb93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.415032 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc72a06ee8e4420b07a17d2feb3fcc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.415104 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 864f2602840c40fc8606b3784e671cba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.415294 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94365c44b5e24acab5df60c67203794a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.415624 28523 tablet_service.cc:1511] Processing CreateTablet for tablet edc72a06ee8e4420b07a17d2feb3fcc2 (DEFAULT_TABLE table=default.Table_08ed798477b446ac8960ab3aa1609b8a [id=9d63ad7327a44124af817b465a9ae0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.415934 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc72a06ee8e4420b07a17d2feb3fcc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.418634 28661 tablet_service.cc:1511] Processing CreateTablet for tablet edc72a06ee8e4420b07a17d2feb3fcc2 (DEFAULT_TABLE table=default.Table_08ed798477b446ac8960ab3aa1609b8a [id=9d63ad7327a44124af817b465a9ae0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.418671 28387 tablet_service.cc:1511] Processing CreateTablet for tablet 0a51e830e3014726b1d1c2608973f036 (DEFAULT_TABLE table=default.Table_696ead97d50747228517c265538b25fb [id=716ae493050c4288b9bf041c94523b05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.418670 28524 tablet_service.cc:1511] Processing CreateTablet for tablet 0a51e830e3014726b1d1c2608973f036 (DEFAULT_TABLE table=default.Table_696ead97d50747228517c265538b25fb [id=716ae493050c4288b9bf041c94523b05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.418959 28661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc72a06ee8e4420b07a17d2feb3fcc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.418969 28387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a51e830e3014726b1d1c2608973f036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.418989 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a51e830e3014726b1d1c2608973f036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.422618 28659 tablet_service.cc:1511] Processing CreateTablet for tablet d6871232860b41d489e34d95eb897241 (DEFAULT_TABLE table=default.Table_c18d75e5991b440caf136e9248a572da [id=5a3cb5befb614a70b4ecf8ccfedf8ecd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.422667 28386 tablet_service.cc:1511] Processing CreateTablet for tablet d6871232860b41d489e34d95eb897241 (DEFAULT_TABLE table=default.Table_c18d75e5991b440caf136e9248a572da [id=5a3cb5befb614a70b4ecf8ccfedf8ecd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.422657 28525 tablet_service.cc:1511] Processing CreateTablet for tablet d6871232860b41d489e34d95eb897241 (DEFAULT_TABLE table=default.Table_c18d75e5991b440caf136e9248a572da [id=5a3cb5befb614a70b4ecf8ccfedf8ecd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.422953 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6871232860b41d489e34d95eb897241. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.422936 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6871232860b41d489e34d95eb897241. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.423036 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6871232860b41d489e34d95eb897241. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.424189 28384 tablet_service.cc:1511] Processing CreateTablet for tablet 125bb6885ab348d885d5a3b4bdd45bc7 (DEFAULT_TABLE table=default.Table_af38438888374cf2b0b8f3db85810efc [id=84cf9477c8584508b8e1c847cdf58b28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.424587 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 125bb6885ab348d885d5a3b4bdd45bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.426659 28527 tablet_service.cc:1511] Processing CreateTablet for tablet 125bb6885ab348d885d5a3b4bdd45bc7 (DEFAULT_TABLE table=default.Table_af38438888374cf2b0b8f3db85810efc [id=84cf9477c8584508b8e1c847cdf58b28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.426956 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 125bb6885ab348d885d5a3b4bdd45bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.427026 28390 tablet_service.cc:1511] Processing CreateTablet for tablet 864f2602840c40fc8606b3784e671cba (DEFAULT_TABLE table=default.Table_0827d01c77434679a1d04c782d175304 [id=3ac9a6b120d44ac2bae74d85d7bc6930]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.427377 28390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 864f2602840c40fc8606b3784e671cba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.428607 28654 tablet_service.cc:1511] Processing CreateTablet for tablet 94365c44b5e24acab5df60c67203794a (DEFAULT_TABLE table=default.Table_e4c2db87fb024027b4a6e457a8524f10 [id=770fade529a64ec29e70968dd6d6bb93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.428934 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94365c44b5e24acab5df60c67203794a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.430779 28519 tablet_service.cc:1511] Processing CreateTablet for tablet da4d4f03911f4ac099ecc2fa1d8af494 (DEFAULT_TABLE table=default.Table_1c0c9338a74e4ac8aea2fdeea2495aec [id=f7cb92d4ab0742af9946f6bcf708df4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.431360 28519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da4d4f03911f4ac099ecc2fa1d8af494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.434649 28658 tablet_service.cc:1511] Processing CreateTablet for tablet 125bb6885ab348d885d5a3b4bdd45bc7 (DEFAULT_TABLE table=default.Table_af38438888374cf2b0b8f3db85810efc [id=84cf9477c8584508b8e1c847cdf58b28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.435006 28658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 125bb6885ab348d885d5a3b4bdd45bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.435845 28381 tablet_service.cc:1511] Processing CreateTablet for tablet 36b8a46a41e543c58a946b6764618c7d (DEFAULT_TABLE table=default.Table_cd35c87466e64da6bd51a0ed413b9933 [id=4961f93e6ad64ec7b3bddcc62db96522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.436475 28381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36b8a46a41e543c58a946b6764618c7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.438117 28382 tablet_service.cc:1511] Processing CreateTablet for tablet da4d4f03911f4ac099ecc2fa1d8af494 (DEFAULT_TABLE table=default.Table_1c0c9338a74e4ac8aea2fdeea2495aec [id=f7cb92d4ab0742af9946f6bcf708df4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.438619 28382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da4d4f03911f4ac099ecc2fa1d8af494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.452332 28799 raft_consensus.cc:359] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.452535 28799 raft_consensus.cc:385] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.453164 28389 tablet_service.cc:1511] Processing CreateTablet for tablet a1a06078974e4bfbbc4b949802a39bf1 (DEFAULT_TABLE table=default.Table_d202b6dfbdee4280bdc0bc6e2c7094be [id=7b747ef3b2494ca78c499c5bb75648d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.453368 28799 raft_consensus.cc:740] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.453785 28799 consensus_queue.cc:260] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.455839 28799 ts_tablet_manager.cc:1434] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.055s user 0.001s sys 0.001s
I20260618 13:59:21.456202 28799 tablet_bootstrap.cc:492] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:21.454216 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1a06078974e4bfbbc4b949802a39bf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.459112 28799 tablet_bootstrap.cc:654] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.463020 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet 102a500268574e428cb2d45e52849f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:21.466434 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet b8389b77e6b8407db38e07e85f7509b1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b8389b77e6b8407db38e07e85f7509b1 already in progress: creating tablet
I20260618 13:59:21.466573 28654 tablet_service.cc:1511] Processing CreateTablet for tablet da4d4f03911f4ac099ecc2fa1d8af494 (DEFAULT_TABLE table=default.Table_1c0c9338a74e4ac8aea2fdeea2495aec [id=f7cb92d4ab0742af9946f6bcf708df4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.466802 28802 tablet_replica.cc:333] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:21.467001 28802 raft_consensus.cc:2243] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:21.467214 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da4d4f03911f4ac099ecc2fa1d8af494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.467423 28802 pending_rounds.cc:70] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b: Trying to abort 1 pending ops.
I20260618 13:59:21.467636 28802 pending_rounds.cc:77] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298216596090904576 op_type: NO_OP noop_request { }
I20260618 13:59:21.467823 28802 raft_consensus.cc:2889] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:21.471053 28802 raft_consensus.cc:2272] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.472380 28802 ts_tablet_manager.cc:1916] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.477459 28807 raft_consensus.cc:493] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:21.478027 28807 raft_consensus.cc:515] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.478678 28807 leader_election.cc:290] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:21.479828 28346 leader_election.cc:343] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 102a500268574e428cb2d45e52849f57
I20260618 13:59:21.481511 28226 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_c18d75e5991b440caf136e9248a572da" } modify_external_catalogs: true
I20260618 13:59:21.481741 28226 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_c18d75e5991b440caf136e9248a572da" } modify_external_catalogs: true
I20260618 13:59:21.482115 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102a500268574e428cb2d45e52849f57" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:21.482223 28799 tablet_bootstrap.cc:492] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.482286 28222 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_af38438888374cf2b0b8f3db85810efc" } modify_external_catalogs: true
I20260618 13:59:21.482375 28799 ts_tablet_manager.cc:1403] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.001s
I20260618 13:59:21.482463 28222 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_af38438888374cf2b0b8f3db85810efc" } modify_external_catalogs: true
W20260618 13:59:21.482782 28346 leader_election.cc:343] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:21.482893 28346 leader_election.cc:304] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:21.483021 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_4cd055d9d2134a539e6a1f8c4741ad60"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:21.483209 28799 raft_consensus.cc:359] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.483294 28807 raft_consensus.cc:2749] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:21.483394 28799 raft_consensus.cc:385] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.483466 28799 raft_consensus.cc:740] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
W20260618 13:59:21.483582 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4cd055d9d2134a539e6a1f8c4741ad60 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.483640 28225 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_85f1769dbcad44ce9a4f98a5493c1530"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:21.483664 28799 consensus_queue.cc:260] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:21.484123 28225 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_85f1769dbcad44ce9a4f98a5493c1530 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.485015 28520 tablet_service.cc:1511] Processing CreateTablet for tablet 36b8a46a41e543c58a946b6764618c7d (DEFAULT_TABLE table=default.Table_cd35c87466e64da6bd51a0ed413b9933 [id=4961f93e6ad64ec7b3bddcc62db96522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.485394 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36b8a46a41e543c58a946b6764618c7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.486055 28799 ts_tablet_manager.cc:1434] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:21.486554 28227 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_f0607bf6529e4f209e00f49908420141" } modify_external_catalogs: true
I20260618 13:59:21.486742 28227 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_f0607bf6529e4f209e00f49908420141" } modify_external_catalogs: true
I20260618 13:59:21.487404 28799 tablet_bootstrap.cc:492] T d5cb3137659a4481840fe04174ef682d P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:21.487365 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_c5c65f038c8f4f2783a705ffcc320629"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:21.487776 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c5c65f038c8f4f2783a705ffcc320629 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.489076 28799 tablet_bootstrap.cc:654] T d5cb3137659a4481840fe04174ef682d P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.489993 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_fac3fdd9b466497683bc00300a85fed1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:21.490406 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fac3fdd9b466497683bc00300a85fed1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.492420 28802 ts_tablet_manager.cc:1929] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:21.492528 28802 log.cc:1199] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/717c38fbd9cb492b9547372e2160f84f
I20260618 13:59:21.492923 28802 ts_tablet_manager.cc:1950] T 717c38fbd9cb492b9547372e2160f84f P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:21.493263 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_be74477d3cd94a6891d879db47d3efd1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:21.493693 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_be74477d3cd94a6891d879db47d3efd1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.494004 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 717c38fbd9cb492b9547372e2160f84f (table default.Table_a2f92d4bf5bc49aca105a2886053d230 [id=3bcc877eddec4d00b3dc1760cc635b36]) successfully deleted
I20260618 13:59:21.495237 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_b49acca2ee444ece953fcedf4b6d28d7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:21.495692 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b49acca2ee444ece953fcedf4b6d28d7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:21.495872 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 31b328882af549a78535cdd55c6ea09c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 31b328882af549a78535cdd55c6ea09c already in progress: creating tablet
W20260618 13:59:21.497007 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 95db5ddaf5d5486988750efed5972eb6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 95db5ddaf5d5486988750efed5972eb6 already in progress: creating tablet
I20260618 13:59:21.497321 28802 tablet_replica.cc:333] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:21.497535 28802 raft_consensus.cc:2243] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.497694 28802 raft_consensus.cc:2272] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.499022 28802 ts_tablet_manager.cc:1916] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.502445 28799 tablet_bootstrap.cc:492] T d5cb3137659a4481840fe04174ef682d P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.502614 28799 ts_tablet_manager.cc:1403] T d5cb3137659a4481840fe04174ef682d P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.003s
I20260618 13:59:21.503463 28799 raft_consensus.cc:359] T d5cb3137659a4481840fe04174ef682d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.503682 28799 raft_consensus.cc:385] T d5cb3137659a4481840fe04174ef682d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.503795 28799 raft_consensus.cc:740] T d5cb3137659a4481840fe04174ef682d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.503995 28229 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 570d36a27b7d417989ad72885d38ff63 (table default.Table_7d3c22c138704d71a7f2a0e756cfa3a2 [id=e38a333ca62d42518ab53a9ac17df2b7]). Expected version 3 got 0
I20260618 13:59:21.504088 28799 consensus_queue.cc:260] T d5cb3137659a4481840fe04174ef682d P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.504654 28229 master_service.cc:960] ReplaceTablet: received request to replace tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:21.507213 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 8b59255b14df4158afe37cb8890e7415 (DEFAULT_TABLE table=default.Table_a2f92d4bf5bc49aca105a2886053d230 [id=3bcc877eddec4d00b3dc1760cc635b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.507622 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b59255b14df4158afe37cb8890e7415. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.400494 28655 tablet_service.cc:1511] Processing CreateTablet for tablet a1a06078974e4bfbbc4b949802a39bf1 (DEFAULT_TABLE table=default.Table_d202b6dfbdee4280bdc0bc6e2c7094be [id=7b747ef3b2494ca78c499c5bb75648d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.512122 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1a06078974e4bfbbc4b949802a39bf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.512233 28385 tablet_service.cc:1511] Processing CreateTablet for tablet d9c00003300246ada2a356e3e1cf684b (DEFAULT_TABLE table=default.Table_a9703f9d3b54453691c354131f824e22 [id=f298b441bd9b444b9edd832be0adb03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.512620 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9c00003300246ada2a356e3e1cf684b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.515061 28799 ts_tablet_manager.cc:1434] T d5cb3137659a4481840fe04174ef682d P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.012s user 0.000s sys 0.002s
I20260618 13:59:21.515899 28799 tablet_bootstrap.cc:492] T 8c9d89fa75174a969ded07e0596db99b P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:21.519186 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 36b8a46a41e543c58a946b6764618c7d (DEFAULT_TABLE table=default.Table_cd35c87466e64da6bd51a0ed413b9933 [id=4961f93e6ad64ec7b3bddcc62db96522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.519562 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36b8a46a41e543c58a946b6764618c7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.520970 28411 raft_consensus.cc:1275] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Refusing update from remote peer d6a169b16ced46ca9cfbc631709bb257: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 13:59:21.521656 28799 tablet_bootstrap.cc:654] T 8c9d89fa75174a969ded07e0596db99b P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.521852 28806 consensus_queue.cc:1048] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:21.523295 28549 raft_consensus.cc:1275] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Refusing update from remote peer d6a169b16ced46ca9cfbc631709bb257: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 13:59:21.523878 28802 ts_tablet_manager.cc:1929] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.524026 28802 log.cc:1199] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/102a500268574e428cb2d45e52849f57
I20260618 13:59:21.524358 28802 ts_tablet_manager.cc:1950] T 102a500268574e428cb2d45e52849f57 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:21.525424 28806 consensus_queue.cc:1048] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:21.525529 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 102a500268574e428cb2d45e52849f57 (table default.Table_bd146d3bdafe4c66b00a25d98887f1e2 [id=76865706d6a54ec198dd205b6ad61920]) successfully deleted
I20260618 13:59:21.530053 28799 tablet_bootstrap.cc:492] T 8c9d89fa75174a969ded07e0596db99b P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.530232 28799 ts_tablet_manager.cc:1403] T 8c9d89fa75174a969ded07e0596db99b P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.005s
I20260618 13:59:21.531306 28799 raft_consensus.cc:359] T 8c9d89fa75174a969ded07e0596db99b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.531499 28799 raft_consensus.cc:385] T 8c9d89fa75174a969ded07e0596db99b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.531692 28799 raft_consensus.cc:740] T 8c9d89fa75174a969ded07e0596db99b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.531888 28656 tablet_service.cc:1511] Processing CreateTablet for tablet 8b59255b14df4158afe37cb8890e7415 (DEFAULT_TABLE table=default.Table_a2f92d4bf5bc49aca105a2886053d230 [id=3bcc877eddec4d00b3dc1760cc635b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.531968 28799 consensus_queue.cc:260] T 8c9d89fa75174a969ded07e0596db99b P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.532503 28799 ts_tablet_manager.cc:1434] T 8c9d89fa75174a969ded07e0596db99b P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:21.532845 28799 tablet_bootstrap.cc:492] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:21.534049 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b59255b14df4158afe37cb8890e7415. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.534544 28799 tablet_bootstrap.cc:654] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.536791 28799 tablet_bootstrap.cc:492] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.536949 28799 ts_tablet_manager.cc:1403] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260618 13:59:21.537889 28799 raft_consensus.cc:359] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.538118 28799 raft_consensus.cc:385] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.538213 28799 raft_consensus.cc:740] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.538514 28799 consensus_queue.cc:260] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.539047 28520 tablet_service.cc:1511] Processing CreateTablet for tablet d9c00003300246ada2a356e3e1cf684b (DEFAULT_TABLE table=default.Table_a9703f9d3b54453691c354131f824e22 [id=f298b441bd9b444b9edd832be0adb03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.539194 28799 ts_tablet_manager.cc:1434] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:21.539276 28654 tablet_service.cc:1511] Processing CreateTablet for tablet d9c00003300246ada2a356e3e1cf684b (DEFAULT_TABLE table=default.Table_a9703f9d3b54453691c354131f824e22 [id=f298b441bd9b444b9edd832be0adb03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.539568 28807 raft_consensus.cc:493] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:21.539669 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9c00003300246ada2a356e3e1cf684b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.539726 28807 raft_consensus.cc:515] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.540302 28807 leader_election.cc:290] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:21.541077 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15bbf81e3fbf4927b822afa0635909eb" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:21.541091 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15bbf81e3fbf4927b822afa0635909eb" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:21.541667 28346 leader_election.cc:343] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:21.542186 28346 leader_election.cc:343] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:21.542313 28346 leader_election.cc:304] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:21.542774 28807 raft_consensus.cc:2749] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:21.543103 28799 tablet_bootstrap.cc:492] T eeffc96b9b97441d9301e4919d0047e1 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:21.544873 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9c00003300246ada2a356e3e1cf684b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.546588 28799 tablet_bootstrap.cc:654] T eeffc96b9b97441d9301e4919d0047e1 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.548919 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 8b59255b14df4158afe37cb8890e7415 (DEFAULT_TABLE table=default.Table_a2f92d4bf5bc49aca105a2886053d230 [id=3bcc877eddec4d00b3dc1760cc635b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.399787 28797 raft_consensus.cc:359] T d5cb3137659a4481840fe04174ef682d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.549366 28797 raft_consensus.cc:385] T d5cb3137659a4481840fe04174ef682d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.549467 28797 raft_consensus.cc:740] T d5cb3137659a4481840fe04174ef682d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.549793 28797 consensus_queue.cc:260] T d5cb3137659a4481840fe04174ef682d P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.550256 28797 ts_tablet_manager.cc:1434] T d5cb3137659a4481840fe04174ef682d P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.151s user 0.000s sys 0.001s
I20260618 13:59:21.550595 28797 tablet_bootstrap.cc:492] T 15bbf81e3fbf4927b822afa0635909eb P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:21.554085 28904 mvcc.cc:204] Tried to move back new op lower bound from 7298216597540753408 to 7298216596250075136. Current Snapshot: MvccSnapshot[applied={T|T < 7298216597540753408}]
I20260618 13:59:21.555413 28221 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 570d36a27b7d417989ad72885d38ff63 (table default.Table_7d3c22c138704d71a7f2a0e756cfa3a2 [id=e38a333ca62d42518ab53a9ac17df2b7]). Expected version 3 got 0
I20260618 13:59:21.555706 28223 master_service.cc:960] ReplaceTablet: received request to replace tablet 059716cec7f94ea6ad8989fecc6f832a from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:21.556308 28799 tablet_bootstrap.cc:492] T eeffc96b9b97441d9301e4919d0047e1 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.556481 28799 ts_tablet_manager.cc:1403] T eeffc96b9b97441d9301e4919d0047e1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.006s
I20260618 13:59:21.556352 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_0c3d584cea7845df83ac7adfe73d31e4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:21.556479 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_88951dd073b74c99aa4a0bd7dcf2e19d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:21.556919 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c3d584cea7845df83ac7adfe73d31e4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.557353 28799 raft_consensus.cc:359] T eeffc96b9b97441d9301e4919d0047e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.557574 28799 raft_consensus.cc:385] T eeffc96b9b97441d9301e4919d0047e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.557650 28799 raft_consensus.cc:740] T eeffc96b9b97441d9301e4919d0047e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.557974 28799 consensus_queue.cc:260] T eeffc96b9b97441d9301e4919d0047e1 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.559139 28799 ts_tablet_manager.cc:1434] T eeffc96b9b97441d9301e4919d0047e1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:21.559474 28799 tablet_bootstrap.cc:492] T 85a3b27d914a412498919a3f65e20522 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:21.559634 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_8a78f34a376f49f88006dc80c596745f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:21.560189 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a78f34a376f49f88006dc80c596745f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.561026 28906 mvcc.cc:204] Tried to move back new op lower bound from 7298216597540753408 to 7298216596250075136. Current Snapshot: MvccSnapshot[applied={T|T < 7298216597540753408}]
I20260618 13:59:21.561264 28799 tablet_bootstrap.cc:654] T 85a3b27d914a412498919a3f65e20522 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.561312 28228 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_5b9198f79dca44a9a66c38929d637bde" } new_table_name: "default.Table_4fd7db2ab84d49febc9b21aa3c03d5cf" modify_external_catalogs: true
I20260618 13:59:21.562129 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b59255b14df4158afe37cb8890e7415. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.562734 28225 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_ab57b782a92747e78e2bfe28480aa0ec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:21.563140 28225 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab57b782a92747e78e2bfe28480aa0ec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.563544 28799 tablet_bootstrap.cc:492] T 85a3b27d914a412498919a3f65e20522 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.563683 28799 ts_tablet_manager.cc:1403] T 85a3b27d914a412498919a3f65e20522 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260618 13:59:21.564252 28220 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_dea82febaef74181863a39cd4c12af41" } new_table_name: "default.Table_af2d1c7150e84f96807cda9b1dabd299" modify_external_catalogs: true
I20260618 13:59:21.564427 28799 raft_consensus.cc:359] T 85a3b27d914a412498919a3f65e20522 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:21.564581 28799 raft_consensus.cc:385] T 85a3b27d914a412498919a3f65e20522 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.564649 28799 raft_consensus.cc:740] T 85a3b27d914a412498919a3f65e20522 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.564815 28799 consensus_queue.cc:260] T 85a3b27d914a412498919a3f65e20522 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:21.565270 28799 ts_tablet_manager.cc:1434] T 85a3b27d914a412498919a3f65e20522 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260618 13:59:21.565487 28799 tablet_bootstrap.cc:492] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:21.556921 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_88951dd073b74c99aa4a0bd7dcf2e19d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.566951 28799 tablet_bootstrap.cc:654] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.571007 28799 tablet_bootstrap.cc:492] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.571189 28799 ts_tablet_manager.cc:1403] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260618 13:59:21.572031 28799 raft_consensus.cc:359] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.572248 28799 raft_consensus.cc:385] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.572335 28799 raft_consensus.cc:740] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.572727 28799 consensus_queue.cc:260] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.573346 28381 tablet_service.cc:1558] Processing DeleteTablet for tablet d5cb3137659a4481840fe04174ef682d with delete_type TABLET_DATA_DELETED (replaced by tablet 68b8293a7c924a449307ab88a1cf9290) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.573437 28799 ts_tablet_manager.cc:1434] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260618 13:59:21.573666 28802 tablet_replica.cc:333] T d5cb3137659a4481840fe04174ef682d P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:21.573788 28801 ts_tablet_manager.cc:1916] T 91c250dfbd72416d9d74f6cbb67b2bdf P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.573815 28802 raft_consensus.cc:2243] T d5cb3137659a4481840fe04174ef682d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.574012 28802 raft_consensus.cc:2272] T d5cb3137659a4481840fe04174ef682d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.574777 28802 ts_tablet_manager.cc:1916] T d5cb3137659a4481840fe04174ef682d P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.575611 28911 tablet.cc:1724] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260618 13:59:21.577489 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet d5cb3137659a4481840fe04174ef682d with delete_type TABLET_DATA_DELETED (replaced by tablet 68b8293a7c924a449307ab88a1cf9290) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.577739 28800 tablet_replica.cc:333] T d5cb3137659a4481840fe04174ef682d P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:21.577904 28800 raft_consensus.cc:2243] T d5cb3137659a4481840fe04174ef682d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.578387 28800 raft_consensus.cc:2272] T d5cb3137659a4481840fe04174ef682d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.578985 28800 ts_tablet_manager.cc:1916] T d5cb3137659a4481840fe04174ef682d P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.582144 28802 ts_tablet_manager.cc:1929] T d5cb3137659a4481840fe04174ef682d P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.583165 28802 log.cc:1199] T d5cb3137659a4481840fe04174ef682d P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/d5cb3137659a4481840fe04174ef682d
I20260618 13:59:21.584239 28802 ts_tablet_manager.cc:1950] T d5cb3137659a4481840fe04174ef682d P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:21.585072 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet d5cb3137659a4481840fe04174ef682d (table default.Table_c97dcabd4c2b45a18f445a099ec7ace9 [id=38bde7a301814aa285e4bd30bcc527aa]) successfully deleted
I20260618 13:59:21.584512 28799 tablet_bootstrap.cc:492] T 5332b6bb76ea419086341738fd135b24 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:21.585703 28800 ts_tablet_manager.cc:1929] T d5cb3137659a4481840fe04174ef682d P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.585821 28800 log.cc:1199] T d5cb3137659a4481840fe04174ef682d P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/d5cb3137659a4481840fe04174ef682d
I20260618 13:59:21.586164 28800 ts_tablet_manager.cc:1950] T d5cb3137659a4481840fe04174ef682d P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:21.587388 28799 tablet_bootstrap.cc:654] T 5332b6bb76ea419086341738fd135b24 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.574978 28910 tablet.cc:1724] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260618 13:59:21.588160 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet d5cb3137659a4481840fe04174ef682d (table default.Table_c97dcabd4c2b45a18f445a099ec7ace9 [id=38bde7a301814aa285e4bd30bcc527aa]) successfully deleted
I20260618 13:59:21.590188 28797 tablet_bootstrap.cc:654] T 15bbf81e3fbf4927b822afa0635909eb P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.599764 28227 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet eeffc96b9b97441d9301e4919d0047e1
I20260618 13:59:21.602697 28226 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet d6871232860b41d489e34d95eb897241
I20260618 13:59:21.603987 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet d6871232860b41d489e34d95eb897241 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:21.604205 28220 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e1a75029992645c5b942c0ec07862bb4 consensus state has no leader
I20260618 13:59:21.604662 28220 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_dd833764ce1041cba71575be46d771a9" } new_table_name: "default.Table_dbc49898f77e441cb501871dfae61db4" modify_external_catalogs: true
I20260618 13:59:21.605391 28222 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 125bb6885ab348d885d5a3b4bdd45bc7
W20260618 13:59:21.605599 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet d6871232860b41d489e34d95eb897241 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d6871232860b41d489e34d95eb897241 already in progress: creating tablet
I20260618 13:59:21.606033 28222 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_67044988023049af9b42c306e291361c" } new_table_name: "default.Table_5f8fbc1f56c1484cb99b55730190bbcf" modify_external_catalogs: true
I20260618 13:59:21.606339 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet 125bb6885ab348d885d5a3b4bdd45bc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.606446 28229 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0
I20260618 13:59:21.606931 28229 catalog_manager.cc:6784] ReplaceTablet: tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 deleted and replaced by tablet ac58e939730c4af681c2f08bcfbb6602
I20260618 13:59:21.609675 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 with delete_type TABLET_DATA_DELETED (replaced by tablet ac58e939730c4af681c2f08bcfbb6602) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:21.609863 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 125bb6885ab348d885d5a3b4bdd45bc7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 125bb6885ab348d885d5a3b4bdd45bc7 already in progress: creating tablet
W20260618 13:59:21.610567 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 already in progress: creating tablet
W20260618 13:59:21.611692 28228 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7e5517e820ae47de810ff817f5382839 consensus state has no leader
I20260618 13:59:21.611945 28229 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_30c1b9baedbf4ed588651ac692eff7b6" } new_table_name: "default.Table_5adadd2ea1314b34a29501870fec65f3" modify_external_catalogs: true
I20260618 13:59:21.615420 28799 tablet_bootstrap.cc:492] T 5332b6bb76ea419086341738fd135b24 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.615586 28799 ts_tablet_manager.cc:1403] T 5332b6bb76ea419086341738fd135b24 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.031s user 0.009s sys 0.018s
I20260618 13:59:21.616361 28799 raft_consensus.cc:359] T 5332b6bb76ea419086341738fd135b24 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.616570 28799 raft_consensus.cc:385] T 5332b6bb76ea419086341738fd135b24 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.616671 28799 raft_consensus.cc:740] T 5332b6bb76ea419086341738fd135b24 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.618777 28227 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 397c7abe26f4451298558d6a0875b677 (table default.Table_2be2c24d66ae4f1d8bf1e78d134841c7 [id=ce780967c12a412a8d7f0bef2f9c00e3]). Expected version 5 got 0
I20260618 13:59:21.621248 28381 tablet_service.cc:1558] Processing DeleteTablet for tablet eeffc96b9b97441d9301e4919d0047e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.621541 28802 tablet_replica.cc:333] T eeffc96b9b97441d9301e4919d0047e1 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:21.621737 28802 raft_consensus.cc:2243] T eeffc96b9b97441d9301e4919d0047e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.621784 28223 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 397c7abe26f4451298558d6a0875b677 (table default.Table_2be2c24d66ae4f1d8bf1e78d134841c7 [id=ce780967c12a412a8d7f0bef2f9c00e3]). Expected version 5 got 0
I20260618 13:59:21.621881 28802 raft_consensus.cc:2272] T eeffc96b9b97441d9301e4919d0047e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.622576 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c9d89fa75174a969ded07e0596db99b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.622704 28802 ts_tablet_manager.cc:1916] T eeffc96b9b97441d9301e4919d0047e1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.622918 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet d5cb3137659a4481840fe04174ef682d with delete_type TABLET_DATA_DELETED (replaced by tablet 68b8293a7c924a449307ab88a1cf9290) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.623042 28799 consensus_queue.cc:260] T 5332b6bb76ea419086341738fd135b24 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.623545 28799 ts_tablet_manager.cc:1434] T 5332b6bb76ea419086341738fd135b24 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260618 13:59:21.624637 28799 tablet_bootstrap.cc:492] T 059716cec7f94ea6ad8989fecc6f832a P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:21.626544 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 15bbf81e3fbf4927b822afa0635909eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.627130 28799 tablet_bootstrap.cc:654] T 059716cec7f94ea6ad8989fecc6f832a P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.633168 28802 ts_tablet_manager.cc:1929] T eeffc96b9b97441d9301e4919d0047e1 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.633312 28802 log.cc:1199] T eeffc96b9b97441d9301e4919d0047e1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/eeffc96b9b97441d9301e4919d0047e1
I20260618 13:59:21.633972 28802 ts_tablet_manager.cc:1950] T eeffc96b9b97441d9301e4919d0047e1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:21.635246 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet eeffc96b9b97441d9301e4919d0047e1 (table default.Table_f0607bf6529e4f209e00f49908420141 [id=972bc2c124ac459da06ec4093c5b564b]) successfully deleted
I20260618 13:59:21.637058 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet eeffc96b9b97441d9301e4919d0047e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.637565 28802 tablet_replica.cc:333] T 8c9d89fa75174a969ded07e0596db99b P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:21.638175 28799 tablet_bootstrap.cc:492] T 059716cec7f94ea6ad8989fecc6f832a P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.638324 28799 ts_tablet_manager.cc:1403] T 059716cec7f94ea6ad8989fecc6f832a P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260618 13:59:21.638667 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet d5cb3137659a4481840fe04174ef682d with delete_type TABLET_DATA_DELETED (replaced by tablet 68b8293a7c924a449307ab88a1cf9290) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.639170 28802 raft_consensus.cc:2243] T 8c9d89fa75174a969ded07e0596db99b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.639509 28802 raft_consensus.cc:2272] T 8c9d89fa75174a969ded07e0596db99b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.639024 28799 raft_consensus.cc:359] T 059716cec7f94ea6ad8989fecc6f832a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.640138 28799 raft_consensus.cc:385] T 059716cec7f94ea6ad8989fecc6f832a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.640231 28799 raft_consensus.cc:740] T 059716cec7f94ea6ad8989fecc6f832a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.640462 28799 consensus_queue.cc:260] T 059716cec7f94ea6ad8989fecc6f832a P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.640936 28799 ts_tablet_manager.cc:1434] T 059716cec7f94ea6ad8989fecc6f832a P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260618 13:59:21.641304 28799 tablet_bootstrap.cc:492] T b8389b77e6b8407db38e07e85f7509b1 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:21.641921 28797 tablet_bootstrap.cc:492] T 15bbf81e3fbf4927b822afa0635909eb P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:21.642059 28797 ts_tablet_manager.cc:1403] T 15bbf81e3fbf4927b822afa0635909eb P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.092s user 0.001s sys 0.002s
I20260618 13:59:21.642613 28802 ts_tablet_manager.cc:1916] T 8c9d89fa75174a969ded07e0596db99b P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.645161 28799 tablet_bootstrap.cc:654] T b8389b77e6b8407db38e07e85f7509b1 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.648936 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet d6871232860b41d489e34d95eb897241 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.649188 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet eeffc96b9b97441d9301e4919d0047e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.649256 28797 raft_consensus.cc:359] T 15bbf81e3fbf4927b822afa0635909eb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.649413 28797 raft_consensus.cc:385] T 15bbf81e3fbf4927b822afa0635909eb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.649519 28797 raft_consensus.cc:740] T 15bbf81e3fbf4927b822afa0635909eb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.649834 28801 ts_tablet_manager.cc:1929] T 91c250dfbd72416d9d74f6cbb67b2bdf P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.649992 28801 log.cc:1199] T 91c250dfbd72416d9d74f6cbb67b2bdf P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/91c250dfbd72416d9d74f6cbb67b2bdf
I20260618 13:59:21.649763 28919 tablet.cc:1724] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260618 13:59:21.650357 28801 ts_tablet_manager.cc:1950] T 91c250dfbd72416d9d74f6cbb67b2bdf P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:21.651259 28798 tablet_bootstrap.cc:492] T 717c38fbd9cb492b9547372e2160f84f P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:21.651456 28798 ts_tablet_manager.cc:1403] T 717c38fbd9cb492b9547372e2160f84f P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.266s user 0.022s sys 0.026s
I20260618 13:59:21.652410 28798 raft_consensus.cc:359] T 717c38fbd9cb492b9547372e2160f84f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.652671 28798 raft_consensus.cc:385] T 717c38fbd9cb492b9547372e2160f84f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.652778 28798 raft_consensus.cc:740] T 717c38fbd9cb492b9547372e2160f84f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.652900 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet 125bb6885ab348d885d5a3b4bdd45bc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.653051 28798 consensus_queue.cc:260] T 717c38fbd9cb492b9547372e2160f84f P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.653587 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet 125bb6885ab348d885d5a3b4bdd45bc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.654013 28801 tablet_replica.cc:333] T 587b82a6da8d4fe48b26070803831d9d P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:21.654565 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet d6871232860b41d489e34d95eb897241 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.654155 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 with delete_type TABLET_DATA_DELETED (replaced by tablet ac58e939730c4af681c2f08bcfbb6602) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.654209 28801 raft_consensus.cc:2243] T 587b82a6da8d4fe48b26070803831d9d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.654474 28798 ts_tablet_manager.cc:1434] T 717c38fbd9cb492b9547372e2160f84f P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:21.657112 28801 raft_consensus.cc:2272] T 587b82a6da8d4fe48b26070803831d9d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.657749 28798 tablet_bootstrap.cc:492] T ee783d2c7e274158b3de2cc186f50903 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:21.659539 28801 ts_tablet_manager.cc:1916] T 587b82a6da8d4fe48b26070803831d9d P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.660056 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 with delete_type TABLET_DATA_DELETED (replaced by tablet ac58e939730c4af681c2f08bcfbb6602) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.661068 28798 tablet_bootstrap.cc:654] T ee783d2c7e274158b3de2cc186f50903 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.663427 28799 tablet_bootstrap.cc:492] T b8389b77e6b8407db38e07e85f7509b1 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.664440 28799 ts_tablet_manager.cc:1403] T b8389b77e6b8407db38e07e85f7509b1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.023s user 0.006s sys 0.012s
I20260618 13:59:21.665156 28799 raft_consensus.cc:359] T b8389b77e6b8407db38e07e85f7509b1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.665334 28799 raft_consensus.cc:385] T b8389b77e6b8407db38e07e85f7509b1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.665426 28799 raft_consensus.cc:740] T b8389b77e6b8407db38e07e85f7509b1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.658103 28797 consensus_queue.cc:260] T 15bbf81e3fbf4927b822afa0635909eb P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.665671 28799 consensus_queue.cc:260] T b8389b77e6b8407db38e07e85f7509b1 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.667894 28802 ts_tablet_manager.cc:1929] T 8c9d89fa75174a969ded07e0596db99b P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.668044 28802 log.cc:1199] T 8c9d89fa75174a969ded07e0596db99b P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/8c9d89fa75174a969ded07e0596db99b
I20260618 13:59:21.668144 28799 ts_tablet_manager.cc:1434] T b8389b77e6b8407db38e07e85f7509b1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 13:59:21.668435 28799 tablet_bootstrap.cc:492] T 1d23b5d2d67945a6ab9591969acb0d27 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:21.668462 28802 ts_tablet_manager.cc:1950] T 8c9d89fa75174a969ded07e0596db99b P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:21.669396 28802 tablet_replica.cc:333] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:21.669607 28802 raft_consensus.cc:2243] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.669711 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 8c9d89fa75174a969ded07e0596db99b (table default.Table_07cff5be28854452b80fdac31e7ecba3 [id=b024b7c54de64df0a4047d8f0381a5c8]) successfully deleted
I20260618 13:59:21.669800 28802 raft_consensus.cc:2272] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:21.669967 28214 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet d5cb3137659a4481840fe04174ef682d because the tablet was not found. No further retry: Not found: Tablet not found: d5cb3137659a4481840fe04174ef682d
I20260618 13:59:21.670318 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 91c250dfbd72416d9d74f6cbb67b2bdf (table default.Table_3b5a67572c914054908f9bde6bb0ee09 [id=95f51150591e45e8a69bfacc9ac17f24]) successfully deleted
W20260618 13:59:21.670542 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 717c38fbd9cb492b9547372e2160f84f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 717c38fbd9cb492b9547372e2160f84f already in progress: creating tablet
I20260618 13:59:21.671738 28797 ts_tablet_manager.cc:1434] T 15bbf81e3fbf4927b822afa0635909eb P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.030s user 0.003s sys 0.000s
I20260618 13:59:21.673162 28797 tablet_bootstrap.cc:492] T 85a3b27d914a412498919a3f65e20522 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:21.674026 28799 tablet_bootstrap.cc:654] T 1d23b5d2d67945a6ab9591969acb0d27 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.674671 28221 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_7700ee6bae434a01b621ad02088cc5f0" } modify_external_catalogs: true
I20260618 13:59:21.674755 28225 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_696ead97d50747228517c265538b25fb" } modify_external_catalogs: true
I20260618 13:59:21.674867 28221 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_7700ee6bae434a01b621ad02088cc5f0" } modify_external_catalogs: true
I20260618 13:59:21.674950 28225 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_696ead97d50747228517c265538b25fb" } modify_external_catalogs: true
W20260618 13:59:21.675268 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet eeffc96b9b97441d9301e4919d0047e1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eeffc96b9b97441d9301e4919d0047e1 already in progress: creating tablet
W20260618 13:59:21.675487 28215 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d5cb3137659a4481840fe04174ef682d because the tablet was not found. No further retry: Not found: Tablet not found: d5cb3137659a4481840fe04174ef682d
I20260618 13:59:21.675572 28224 master_service.cc:960] ReplaceTablet: received request to replace tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:21.675454 28228 master_service.cc:960] ReplaceTablet: received request to replace tablet 8be4374ae7cf4907940ae90299bf9ba0 from {username='slave'} at 127.0.0.1:45138
W20260618 13:59:21.675875 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 125bb6885ab348d885d5a3b4bdd45bc7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 125bb6885ab348d885d5a3b4bdd45bc7 already in progress: creating tablet
W20260618 13:59:21.676070 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d6871232860b41d489e34d95eb897241 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d6871232860b41d489e34d95eb897241 already in progress: creating tablet
W20260618 13:59:21.676250 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 already in progress: creating tablet
I20260618 13:59:21.677827 28797 tablet_bootstrap.cc:654] T 85a3b27d914a412498919a3f65e20522 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.679525 28802 ts_tablet_manager.cc:1916] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.679873 28799 tablet_bootstrap.cc:492] T 1d23b5d2d67945a6ab9591969acb0d27 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.680037 28799 ts_tablet_manager.cc:1403] T 1d23b5d2d67945a6ab9591969acb0d27 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.005s
I20260618 13:59:21.680823 28799 raft_consensus.cc:359] T 1d23b5d2d67945a6ab9591969acb0d27 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.681000 28799 raft_consensus.cc:385] T 1d23b5d2d67945a6ab9591969acb0d27 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.681078 28799 raft_consensus.cc:740] T 1d23b5d2d67945a6ab9591969acb0d27 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.681342 28799 consensus_queue.cc:260] T 1d23b5d2d67945a6ab9591969acb0d27 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.681826 28799 ts_tablet_manager.cc:1434] T 1d23b5d2d67945a6ab9591969acb0d27 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 13:59:21.682152 28799 tablet_bootstrap.cc:492] T d6e3060118e844559803a649224d7474 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:21.683302 28220 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 311ba4dd83984f379c8f4f8274bc26c1 consensus state has no leader
I20260618 13:59:21.683570 28220 master_service.cc:960] ReplaceTablet: received request to replace tablet 668c6c6f2034484ca5ccdd51c22dea1d from {username='slave'} at 127.0.0.1:45138
W20260618 13:59:21.683954 28229 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dca54111fea047d9ae0fabe376d89382 consensus state has no leader
I20260618 13:59:21.685717 28799 tablet_bootstrap.cc:654] T d6e3060118e844559803a649224d7474 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.691612 28227 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 29bf263d20934ed6a6efb4b215252c00 (table default.Table_67044988023049af9b42c306e291361c [id=7923680bde144e3dbb78c922d09540b9]). Expected version 3 got 0
W20260618 13:59:21.692440 28222 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 29bf263d20934ed6a6efb4b215252c00 consensus state has no leader
I20260618 13:59:21.693396 28802 ts_tablet_manager.cc:1929] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.693586 28802 log.cc:1199] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/15bbf81e3fbf4927b822afa0635909eb
I20260618 13:59:21.693958 28802 ts_tablet_manager.cc:1950] T 15bbf81e3fbf4927b822afa0635909eb P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:21.694505 28226 catalog_manager.cc:6241] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: default.Table_2be2c24d66ae4f1d8bf1e78d134841c7 [id=ce780967c12a412a8d7f0bef2f9c00e3] alter complete (version 5)
I20260618 13:59:21.695851 28799 tablet_bootstrap.cc:492] T d6e3060118e844559803a649224d7474 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.695998 28799 ts_tablet_manager.cc:1403] T d6e3060118e844559803a649224d7474 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20260618 13:59:21.696768 28799 raft_consensus.cc:359] T d6e3060118e844559803a649224d7474 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.696949 28799 raft_consensus.cc:385] T d6e3060118e844559803a649224d7474 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.697046 28799 raft_consensus.cc:740] T d6e3060118e844559803a649224d7474 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.697280 28799 consensus_queue.cc:260] T d6e3060118e844559803a649224d7474 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.697784 28799 ts_tablet_manager.cc:1434] T d6e3060118e844559803a649224d7474 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:21.698207 28799 tablet_bootstrap.cc:492] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:21.698642 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 15bbf81e3fbf4927b822afa0635909eb (table default.Table_1fd86f80c8fc4490aaadc1278d20df3a [id=e76fba0e75f74f8a9d0a94ff9d7f7b53]) successfully deleted
I20260618 13:59:21.699095 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet 85a3b27d914a412498919a3f65e20522 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.699241 28381 tablet_service.cc:1511] Processing CreateTablet for tablet 1f81428f804a485c967026c4527fd250 (DEFAULT_TABLE table=default.Table_b49acca2ee444ece953fcedf4b6d28d7 [id=b817db93d50144d7b7c238550eb53b10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.699309 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet eeffc96b9b97441d9301e4919d0047e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.699622 28381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f81428f804a485c967026c4527fd250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.700798 28802 tablet_replica.cc:333] T 85a3b27d914a412498919a3f65e20522 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:21.702324 28802 raft_consensus.cc:2243] T 85a3b27d914a412498919a3f65e20522 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.705086 28389 tablet_service.cc:1511] Processing CreateTablet for tablet c552d1316e3340f4bcfa39618ae309b8 (DEFAULT_TABLE table=default.Table_be74477d3cd94a6891d879db47d3efd1 [id=aec77306ae6d47ec91b286291c3ab664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.705358 28802 raft_consensus.cc:2272] T 85a3b27d914a412498919a3f65e20522 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.707244 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c552d1316e3340f4bcfa39618ae309b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.710100 28802 ts_tablet_manager.cc:1916] T 85a3b27d914a412498919a3f65e20522 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.712594 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 4c2b924a096b479cad31c29bf7c99172 (DEFAULT_TABLE table=default.Table_85f1769dbcad44ce9a4f98a5493c1530 [id=431a660bfbed46248a603dc7781cf543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.713994 28385 tablet_service.cc:1511] Processing CreateTablet for tablet c84189a4b1194c87bbdf81f6e0ed56e4 (DEFAULT_TABLE table=default.Table_4cd055d9d2134a539e6a1f8c4741ad60 [id=abeeb5e876424a1cb99b2a09083c8230]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.715660 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c2b924a096b479cad31c29bf7c99172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.715852 28382 tablet_service.cc:1511] Processing CreateTablet for tablet 2188ebf1727240cd886ba7662f8fcf43 (DEFAULT_TABLE table=default.Table_fac3fdd9b466497683bc00300a85fed1 [id=a4ac38531af34b4681abba96b42ffb7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.716166 28391 tablet_service.cc:1511] Processing CreateTablet for tablet 516421cd843b44769efa2b555697c065 (DEFAULT_TABLE table=default.Table_c5c65f038c8f4f2783a705ffcc320629 [id=b06c685b8107494392219d12540a3a6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.716861 28799 tablet_bootstrap.cc:654] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.717300 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 1f81428f804a485c967026c4527fd250 (DEFAULT_TABLE table=default.Table_b49acca2ee444ece953fcedf4b6d28d7 [id=b817db93d50144d7b7c238550eb53b10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.717904 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f81428f804a485c967026c4527fd250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.719741 28660 tablet_service.cc:1511] Processing CreateTablet for tablet c552d1316e3340f4bcfa39618ae309b8 (DEFAULT_TABLE table=default.Table_be74477d3cd94a6891d879db47d3efd1 [id=aec77306ae6d47ec91b286291c3ab664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.720230 28655 tablet_service.cc:1511] Processing CreateTablet for tablet 4c2b924a096b479cad31c29bf7c99172 (DEFAULT_TABLE table=default.Table_85f1769dbcad44ce9a4f98a5493c1530 [id=431a660bfbed46248a603dc7781cf543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.720690 28656 tablet_service.cc:1511] Processing CreateTablet for tablet 1f81428f804a485c967026c4527fd250 (DEFAULT_TABLE table=default.Table_b49acca2ee444ece953fcedf4b6d28d7 [id=b817db93d50144d7b7c238550eb53b10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.721257 28654 tablet_service.cc:1558] Processing DeleteTablet for tablet 15bbf81e3fbf4927b822afa0635909eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.721509 28800 tablet_replica.cc:333] T 15bbf81e3fbf4927b822afa0635909eb P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:21.721766 28800 raft_consensus.cc:2243] T 15bbf81e3fbf4927b822afa0635909eb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.721841 28798 tablet_bootstrap.cc:492] T ee783d2c7e274158b3de2cc186f50903 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:21.721937 28800 raft_consensus.cc:2272] T 15bbf81e3fbf4927b822afa0635909eb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.722002 28798 ts_tablet_manager.cc:1403] T ee783d2c7e274158b3de2cc186f50903 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.064s user 0.012s sys 0.008s
I20260618 13:59:21.722381 28654 tablet_service.cc:1511] Processing CreateTablet for tablet c84189a4b1194c87bbdf81f6e0ed56e4 (DEFAULT_TABLE table=default.Table_4cd055d9d2134a539e6a1f8c4741ad60 [id=abeeb5e876424a1cb99b2a09083c8230]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.720296 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c552d1316e3340f4bcfa39618ae309b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.723085 28650 tablet_service.cc:1511] Processing CreateTablet for tablet 516421cd843b44769efa2b555697c065 (DEFAULT_TABLE table=default.Table_c5c65f038c8f4f2783a705ffcc320629 [id=b06c685b8107494392219d12540a3a6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.723508 28650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 516421cd843b44769efa2b555697c065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.723733 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c2b924a096b479cad31c29bf7c99172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.724133 28651 tablet_service.cc:1511] Processing CreateTablet for tablet 2188ebf1727240cd886ba7662f8fcf43 (DEFAULT_TABLE table=default.Table_fac3fdd9b466497683bc00300a85fed1 [id=a4ac38531af34b4681abba96b42ffb7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.724571 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2188ebf1727240cd886ba7662f8fcf43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.727849 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c84189a4b1194c87bbdf81f6e0ed56e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.729808 28221 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 4a01dfb4d59549399c94640f16045555
I20260618 13:59:21.730515 28220 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 668c6c6f2034484ca5ccdd51c22dea1d
I20260618 13:59:21.730998 28220 catalog_manager.cc:6784] ReplaceTablet: tablet 668c6c6f2034484ca5ccdd51c22dea1d deleted and replaced by tablet 9da90366cb7442bb9f4bf3bab07db070
I20260618 13:59:21.731359 28225 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 0a51e830e3014726b1d1c2608973f036
I20260618 13:59:21.732088 28525 tablet_service.cc:1511] Processing CreateTablet for tablet 4c2b924a096b479cad31c29bf7c99172 (DEFAULT_TABLE table=default.Table_85f1769dbcad44ce9a4f98a5493c1530 [id=431a660bfbed46248a603dc7781cf543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.732584 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c2b924a096b479cad31c29bf7c99172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.733834 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet 4a01dfb4d59549399c94640f16045555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.734723 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 516421cd843b44769efa2b555697c065 (DEFAULT_TABLE table=default.Table_c5c65f038c8f4f2783a705ffcc320629 [id=b06c685b8107494392219d12540a3a6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.734769 28522 tablet_service.cc:1511] Processing CreateTablet for tablet c84189a4b1194c87bbdf81f6e0ed56e4 (DEFAULT_TABLE table=default.Table_4cd055d9d2134a539e6a1f8c4741ad60 [id=abeeb5e876424a1cb99b2a09083c8230]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.735073 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 516421cd843b44769efa2b555697c065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.735242 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet 668c6c6f2034484ca5ccdd51c22dea1d with delete_type TABLET_DATA_DELETED (replaced by tablet 9da90366cb7442bb9f4bf3bab07db070) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.735282 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet 717c38fbd9cb492b9547372e2160f84f with delete_type TABLET_DATA_DELETED (replaced by tablet 8b59255b14df4158afe37cb8890e7415) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.735767 28524 tablet_service.cc:1511] Processing CreateTablet for tablet 2188ebf1727240cd886ba7662f8fcf43 (DEFAULT_TABLE table=default.Table_fac3fdd9b466497683bc00300a85fed1 [id=a4ac38531af34b4681abba96b42ffb7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.736173 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2188ebf1727240cd886ba7662f8fcf43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.737919 28661 tablet_service.cc:1558] Processing DeleteTablet for tablet 4a01dfb4d59549399c94640f16045555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.738188 28659 tablet_service.cc:1558] Processing DeleteTablet for tablet 668c6c6f2034484ca5ccdd51c22dea1d with delete_type TABLET_DATA_DELETED (replaced by tablet 9da90366cb7442bb9f4bf3bab07db070) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.738411 28658 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a51e830e3014726b1d1c2608973f036 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.739080 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f81428f804a485c967026c4527fd250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.742635 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c84189a4b1194c87bbdf81f6e0ed56e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.743505 28382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2188ebf1727240cd886ba7662f8fcf43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.745007 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 516421cd843b44769efa2b555697c065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.747007 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet b8389b77e6b8407db38e07e85f7509b1 with delete_type TABLET_DATA_DELETED (replaced by tablet d9c00003300246ada2a356e3e1cf684b) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.747074 28800 ts_tablet_manager.cc:1916] T 15bbf81e3fbf4927b822afa0635909eb P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.747220 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c84189a4b1194c87bbdf81f6e0ed56e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.754882 28798 raft_consensus.cc:359] T ee783d2c7e274158b3de2cc186f50903 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.755134 28798 raft_consensus.cc:385] T ee783d2c7e274158b3de2cc186f50903 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.755229 28798 raft_consensus.cc:740] T ee783d2c7e274158b3de2cc186f50903 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.755445 28798 consensus_queue.cc:260] T ee783d2c7e274158b3de2cc186f50903 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.755976 28798 ts_tablet_manager.cc:1434] T ee783d2c7e274158b3de2cc186f50903 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.034s user 0.000s sys 0.000s
W20260618 13:59:21.756652 28214 catalog_manager.cc:5100] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): alter failed for tablet 717c38fbd9cb492b9547372e2160f84f (table default.Table_a2f92d4bf5bc49aca105a2886053d230 [id=3bcc877eddec4d00b3dc1760cc635b36]),no further retry: Not found: Tablet not found: 717c38fbd9cb492b9547372e2160f84f
I20260618 13:59:21.758846 28520 tablet_service.cc:1511] Processing CreateTablet for tablet c552d1316e3340f4bcfa39618ae309b8 (DEFAULT_TABLE table=default.Table_be74477d3cd94a6891d879db47d3efd1 [id=aec77306ae6d47ec91b286291c3ab664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.763513 28388 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a51e830e3014726b1d1c2608973f036 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.763912 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a51e830e3014726b1d1c2608973f036 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.764607 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 668c6c6f2034484ca5ccdd51c22dea1d with delete_type TABLET_DATA_DELETED (replaced by tablet 9da90366cb7442bb9f4bf3bab07db070) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.767055 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c552d1316e3340f4bcfa39618ae309b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.770606 28387 tablet_service.cc:1558] Processing DeleteTablet for tablet 4a01dfb4d59549399c94640f16045555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.773447 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d23b5d2d67945a6ab9591969acb0d27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.773923 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 059716cec7f94ea6ad8989fecc6f832a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.774082 28388 tablet_service.cc:1558] Processing DeleteTablet for tablet d6e3060118e844559803a649224d7474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.774883 28800 ts_tablet_manager.cc:1929] T 15bbf81e3fbf4927b822afa0635909eb P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.775029 28800 log.cc:1199] T 15bbf81e3fbf4927b822afa0635909eb P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/15bbf81e3fbf4927b822afa0635909eb
I20260618 13:59:21.776144 28798 tablet_bootstrap.cc:492] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:21.778445 28800 ts_tablet_manager.cc:1950] T 15bbf81e3fbf4927b822afa0635909eb P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:21.779582 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 15bbf81e3fbf4927b822afa0635909eb (table default.Table_1fd86f80c8fc4490aaadc1278d20df3a [id=e76fba0e75f74f8a9d0a94ff9d7f7b53]) successfully deleted
W20260618 13:59:21.780062 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 4a01dfb4d59549399c94640f16045555 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a01dfb4d59549399c94640f16045555 already in progress: creating tablet
I20260618 13:59:21.788095 28802 ts_tablet_manager.cc:1929] T 85a3b27d914a412498919a3f65e20522 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.788247 28802 log.cc:1199] T 85a3b27d914a412498919a3f65e20522 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/85a3b27d914a412498919a3f65e20522
I20260618 13:59:21.788621 28802 ts_tablet_manager.cc:1950] T 85a3b27d914a412498919a3f65e20522 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:21.789638 28802 tablet_replica.cc:333] T b8389b77e6b8407db38e07e85f7509b1 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:21.789671 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 85a3b27d914a412498919a3f65e20522 (table default.Table_94d9d315c21e4cfb9950d96262c9b6e3 [id=a874707bba8d41e7b598d7ad4a3e21f3]) successfully deleted
I20260618 13:59:21.789841 28802 raft_consensus.cc:2243] T b8389b77e6b8407db38e07e85f7509b1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.790017 28802 raft_consensus.cc:2272] T b8389b77e6b8407db38e07e85f7509b1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:21.790136 28214 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet eeffc96b9b97441d9301e4919d0047e1 because the tablet was not found. No further retry: Not found: Tablet not found: eeffc96b9b97441d9301e4919d0047e1
I20260618 13:59:21.794816 28802 ts_tablet_manager.cc:1916] T b8389b77e6b8407db38e07e85f7509b1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.799152 28798 tablet_bootstrap.cc:654] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.803306 28222 master_service.cc:960] ReplaceTablet: received request to replace tablet 125bb6885ab348d885d5a3b4bdd45bc7 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:21.805876 28802 ts_tablet_manager.cc:1929] T b8389b77e6b8407db38e07e85f7509b1 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.806237 28802 log.cc:1199] T b8389b77e6b8407db38e07e85f7509b1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/b8389b77e6b8407db38e07e85f7509b1
I20260618 13:59:21.806859 28802 ts_tablet_manager.cc:1950] T b8389b77e6b8407db38e07e85f7509b1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:21.809307 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet b8389b77e6b8407db38e07e85f7509b1 (table default.Table_a9703f9d3b54453691c354131f824e22 [id=f298b441bd9b444b9edd832be0adb03a]) successfully deleted
W20260618 13:59:21.809863 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 0a51e830e3014726b1d1c2608973f036 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a51e830e3014726b1d1c2608973f036 already in progress: creating tablet
W20260618 13:59:21.810209 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 668c6c6f2034484ca5ccdd51c22dea1d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 668c6c6f2034484ca5ccdd51c22dea1d already in progress: creating tablet
I20260618 13:59:21.810568 28802 tablet_replica.cc:333] T 1d23b5d2d67945a6ab9591969acb0d27 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
W20260618 13:59:21.810802 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 4a01dfb4d59549399c94640f16045555 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a01dfb4d59549399c94640f16045555 already in progress: creating tablet
I20260618 13:59:21.810851 28802 raft_consensus.cc:2243] T 1d23b5d2d67945a6ab9591969acb0d27 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.811329 28802 raft_consensus.cc:2272] T 1d23b5d2d67945a6ab9591969acb0d27 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.812529 28802 ts_tablet_manager.cc:1916] T 1d23b5d2d67945a6ab9591969acb0d27 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.822539 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_7e5df99a84c84d66b548875d6d934656"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:21.823171 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e5df99a84c84d66b548875d6d934656 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.824741 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_ba909970a26742b4b1006b747874b417"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:21.825356 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba909970a26742b4b1006b747874b417 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.826992 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_f5abc4ffe2a848fb8805a523adf0f735"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:21.827538 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f5abc4ffe2a848fb8805a523adf0f735 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:21.827977 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 668c6c6f2034484ca5ccdd51c22dea1d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 668c6c6f2034484ca5ccdd51c22dea1d already in progress: creating tablet
I20260618 13:59:21.828312 28802 ts_tablet_manager.cc:1929] T 1d23b5d2d67945a6ab9591969acb0d27 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.828434 28802 log.cc:1199] T 1d23b5d2d67945a6ab9591969acb0d27 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/1d23b5d2d67945a6ab9591969acb0d27
I20260618 13:59:21.828691 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_f92add1c52b049f183400eb9a01c1928"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:21.828859 28802 ts_tablet_manager.cc:1950] T 1d23b5d2d67945a6ab9591969acb0d27 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
W20260618 13:59:21.828858 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 0a51e830e3014726b1d1c2608973f036 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a51e830e3014726b1d1c2608973f036 already in progress: creating tablet
W20260618 13:59:21.829155 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f92add1c52b049f183400eb9a01c1928 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.829928 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 1d23b5d2d67945a6ab9591969acb0d27 (table default.Table_48dff8bdaa4e430b8850f8d688195bcf [id=fa4cd457fb154247acbfc842852eef04]) successfully deleted
I20260618 13:59:21.830662 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_c7d24bbb3b724c98ba3da942a84843e1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:21.831112 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c7d24bbb3b724c98ba3da942a84843e1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.832125 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet ee783d2c7e274158b3de2cc186f50903 with delete_type TABLET_DATA_DELETED (replaced by tablet 311ba4dd83984f379c8f4f8274bc26c1) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.832494 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_1333d91d141a46acac33460026215195"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:21.832911 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1333d91d141a46acac33460026215195 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.835307 28799 tablet_bootstrap.cc:492] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.835376 28798 tablet_bootstrap.cc:492] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:21.835475 28799 ts_tablet_manager.cc:1403] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.137s user 0.019s sys 0.018s
I20260618 13:59:21.835597 28798 ts_tablet_manager.cc:1403] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.070s user 0.006s sys 0.000s
I20260618 13:59:21.836298 28799 raft_consensus.cc:359] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.836503 28799 raft_consensus.cc:385] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.836585 28799 raft_consensus.cc:740] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.836822 28799 consensus_queue.cc:260] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.838654 28798 raft_consensus.cc:359] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.839811 28222 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_a6146ff7daf149b4bca36d8f09293605" } modify_external_catalogs: true
I20260618 13:59:21.839990 28222 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_a6146ff7daf149b4bca36d8f09293605" } modify_external_catalogs: true
I20260618 13:59:21.840147 28228 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 8be4374ae7cf4907940ae90299bf9ba0
I20260618 13:59:21.840647 28228 catalog_manager.cc:6784] ReplaceTablet: tablet 8be4374ae7cf4907940ae90299bf9ba0 deleted and replaced by tablet 092a9aba9c6e48f5be9cf5d55ba93678
I20260618 13:59:21.841940 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet 8be4374ae7cf4907940ae90299bf9ba0 with delete_type TABLET_DATA_DELETED (replaced by tablet 092a9aba9c6e48f5be9cf5d55ba93678) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.842384 28381 tablet_service.cc:1558] Processing DeleteTablet for tablet 8be4374ae7cf4907940ae90299bf9ba0 with delete_type TABLET_DATA_DELETED (replaced by tablet 092a9aba9c6e48f5be9cf5d55ba93678) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:21.842677 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 8be4374ae7cf4907940ae90299bf9ba0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8be4374ae7cf4907940ae90299bf9ba0 already in progress: creating tablet
I20260618 13:59:21.838865 28798 raft_consensus.cc:385] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.843536 28798 raft_consensus.cc:740] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.843941 28798 consensus_queue.cc:260] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:21.844192 28799 ts_tablet_manager.cc:1434] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260618 13:59:21.845306 28799 tablet_bootstrap.cc:492] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:21.849388 28798 ts_tablet_manager.cc:1434] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.014s user 0.003s sys 0.000s
I20260618 13:59:21.849687 28802 tablet_replica.cc:333] T 059716cec7f94ea6ad8989fecc6f832a P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:21.852011 28802 raft_consensus.cc:2243] T 059716cec7f94ea6ad8989fecc6f832a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.852398 28802 raft_consensus.cc:2272] T 059716cec7f94ea6ad8989fecc6f832a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.850323 28799 tablet_bootstrap.cc:654] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.854087 28802 ts_tablet_manager.cc:1916] T 059716cec7f94ea6ad8989fecc6f832a P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.856643 28227 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_08ed798477b446ac8960ab3aa1609b8a" } new_table_name: "default.Table_cf80fd5bbdde4cfb855c8fbe860a15f3" modify_external_catalogs: true
I20260618 13:59:21.857806 28224 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_a9703f9d3b54453691c354131f824e22" } modify_external_catalogs: true
I20260618 13:59:21.857956 28224 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_a9703f9d3b54453691c354131f824e22" } modify_external_catalogs: true
I20260618 13:59:21.858354 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet 144aa8984a44410fb3f90dc99e9b2274 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.858597 28228 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_e9839256914f48e6bb6480c5db7eb379" } modify_external_catalogs: true
I20260618 13:59:21.858736 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet 8be4374ae7cf4907940ae90299bf9ba0 with delete_type TABLET_DATA_DELETED (replaced by tablet 092a9aba9c6e48f5be9cf5d55ba93678) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.858762 28228 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_e9839256914f48e6bb6480c5db7eb379" } modify_external_catalogs: true
I20260618 13:59:21.859409 28798 tablet_bootstrap.cc:492] T a2ca1e22c38e49d793d413e9eaaac06f P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:21.861440 28797 tablet_bootstrap.cc:492] T 85a3b27d914a412498919a3f65e20522 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:21.861637 28797 ts_tablet_manager.cc:1403] T 85a3b27d914a412498919a3f65e20522 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.190s user 0.041s sys 0.060s
I20260618 13:59:21.861786 28801 ts_tablet_manager.cc:1929] T 587b82a6da8d4fe48b26070803831d9d P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.861908 28801 log.cc:1199] T 587b82a6da8d4fe48b26070803831d9d P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/587b82a6da8d4fe48b26070803831d9d
I20260618 13:59:21.862705 28801 ts_tablet_manager.cc:1950] T 587b82a6da8d4fe48b26070803831d9d P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:21.862905 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_b5800aca80594ec1bd3a86ec47dbfc64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:21.862994 28797 raft_consensus.cc:359] T 85a3b27d914a412498919a3f65e20522 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:21.863188 28797 raft_consensus.cc:385] T 85a3b27d914a412498919a3f65e20522 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.863260 28797 raft_consensus.cc:740] T 85a3b27d914a412498919a3f65e20522 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
W20260618 13:59:21.863443 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5800aca80594ec1bd3a86ec47dbfc64 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.863497 28797 consensus_queue.cc:260] T 85a3b27d914a412498919a3f65e20522 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:21.865370 28229 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_45297e5c8f754fbbb3ab4dac6de1e136"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:21.865813 28229 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_45297e5c8f754fbbb3ab4dac6de1e136 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.866780 28797 ts_tablet_manager.cc:1434] T 85a3b27d914a412498919a3f65e20522 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260618 13:59:21.867246 28797 tablet_bootstrap.cc:492] T eeffc96b9b97441d9301e4919d0047e1 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:21.867358 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_dc44ae6b83d844b6a16ce10ba6dce0ad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:21.863905 28798 tablet_bootstrap.cc:654] T a2ca1e22c38e49d793d413e9eaaac06f P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:21.867770 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc44ae6b83d844b6a16ce10ba6dce0ad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.868878 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 587b82a6da8d4fe48b26070803831d9d (table default.Table_e7787f99590e428ca58ae371cdfeffd5 [id=262236d66ee24ad8a9e4e27b4b1c3ac1]) successfully deleted
I20260618 13:59:21.869072 28797 tablet_bootstrap.cc:654] T eeffc96b9b97441d9301e4919d0047e1 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.869545 28223 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_a2f92d4bf5bc49aca105a2886053d230" } new_table_name: "default.Table_6703d889c9814a9a81e4c5af4e6745f5" modify_external_catalogs: true
W20260618 13:59:21.869609 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 31b328882af549a78535cdd55c6ea09c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 31b328882af549a78535cdd55c6ea09c already in progress: creating tablet
W20260618 13:59:21.870535 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 95db5ddaf5d5486988750efed5972eb6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 95db5ddaf5d5486988750efed5972eb6 already in progress: creating tablet
W20260618 13:59:21.871786 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d6871232860b41d489e34d95eb897241 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d6871232860b41d489e34d95eb897241 already in progress: creating tablet
W20260618 13:59:21.873075 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet eeffc96b9b97441d9301e4919d0047e1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eeffc96b9b97441d9301e4919d0047e1 already in progress: creating tablet
I20260618 13:59:21.877664 28802 ts_tablet_manager.cc:1929] T 059716cec7f94ea6ad8989fecc6f832a P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.879973 28802 log.cc:1199] T 059716cec7f94ea6ad8989fecc6f832a P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/059716cec7f94ea6ad8989fecc6f832a
I20260618 13:59:21.880373 28802 ts_tablet_manager.cc:1950] T 059716cec7f94ea6ad8989fecc6f832a P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:21.881238 28802 tablet_replica.cc:333] T d6e3060118e844559803a649224d7474 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:21.881424 28802 raft_consensus.cc:2243] T d6e3060118e844559803a649224d7474 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.881492 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 059716cec7f94ea6ad8989fecc6f832a (table default.Table_e0177aee52ca4680a5786ad73020ad75 [id=591e2be369c347e883141ed55927044d]) successfully deleted
I20260618 13:59:21.881596 28802 raft_consensus.cc:2272] T d6e3060118e844559803a649224d7474 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.883772 28799 tablet_bootstrap.cc:492] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:21.883746 28802 ts_tablet_manager.cc:1916] T d6e3060118e844559803a649224d7474 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.883927 28799 ts_tablet_manager.cc:1403] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.004s
I20260618 13:59:21.884366 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_c1c5efdc5332431fbb8eef6bd34961eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:21.884737 28799 raft_consensus.cc:359] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.884924 28799 raft_consensus.cc:385] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.885033 28799 raft_consensus.cc:740] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
W20260618 13:59:21.885095 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c1c5efdc5332431fbb8eef6bd34961eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:21.885291 28799 consensus_queue.cc:260] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.886314 28799 ts_tablet_manager.cc:1434] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:21.886400 28807 raft_consensus.cc:493] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:21.886644 28799 tablet_bootstrap.cc:492] T 31b328882af549a78535cdd55c6ea09c P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:21.886639 28807 raft_consensus.cc:515] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.887596 28807 leader_election.cc:290] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:21.888748 28799 tablet_bootstrap.cc:654] T 31b328882af549a78535cdd55c6ea09c P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:21.888769 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e5517e820ae47de810ff817f5382839" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:21.889605 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e5517e820ae47de810ff817f5382839" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:21.889750 28346 leader_election.cc:343] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:21.892493 28346 leader_election.cc:343] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:21.892660 28346 leader_election.cc:304] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:21.893441 28807 raft_consensus.cc:2749] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:21.893751 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet 85a3b27d914a412498919a3f65e20522 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.894007 28800 tablet_replica.cc:333] T 85a3b27d914a412498919a3f65e20522 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:21.894193 28800 raft_consensus.cc:2243] T 85a3b27d914a412498919a3f65e20522 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:21.894390 28800 raft_consensus.cc:2272] T 85a3b27d914a412498919a3f65e20522 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:21.895012 28222 catalog_manager.cc:5978] Not sending DeleteTablet requests; no consensus state for tablet 9da90366cb7442bb9f4bf3bab07db070
I20260618 13:59:21.895121 28800 ts_tablet_manager.cc:1916] T 85a3b27d914a412498919a3f65e20522 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:21.896173 28225 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 7e5517e820ae47de810ff817f5382839 (table default.Table_4fd7db2ab84d49febc9b21aa3c03d5cf [id=49576dabdecf41aca1b15ff3fc0c113c]). Expected version 2 got 0
I20260618 13:59:21.897080 28225 master_service.cc:960] ReplaceTablet: received request to replace tablet c552d1316e3340f4bcfa39618ae309b8 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:21.900568 28382 tablet_service.cc:1511] Processing CreateTablet for tablet 9a0691de253b4b008e5d995fd6b876c2 (DEFAULT_TABLE table=default.Table_ab57b782a92747e78e2bfe28480aa0ec [id=28d43fd5660f459a9acfab2fc0dfbdb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.900605 28381 tablet_service.cc:1511] Processing CreateTablet for tablet 37a6764a5e024903848561c9935b1a43 (DEFAULT_TABLE table=default.Table_88951dd073b74c99aa4a0bd7dcf2e19d [id=51cf184bc7634141841001c082262441]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.900987 28382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a0691de253b4b008e5d995fd6b876c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.901050 28800 ts_tablet_manager.cc:1929] T 85a3b27d914a412498919a3f65e20522 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:21.901264 28800 log.cc:1199] T 85a3b27d914a412498919a3f65e20522 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/85a3b27d914a412498919a3f65e20522
I20260618 13:59:21.901283 28381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a6764a5e024903848561c9935b1a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.901715 28800 ts_tablet_manager.cc:1950] T 85a3b27d914a412498919a3f65e20522 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:21.902765 28656 tablet_service.cc:1511] Processing CreateTablet for tablet 9a0691de253b4b008e5d995fd6b876c2 (DEFAULT_TABLE table=default.Table_ab57b782a92747e78e2bfe28480aa0ec [id=28d43fd5660f459a9acfab2fc0dfbdb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.902858 28650 tablet_service.cc:1511] Processing CreateTablet for tablet 37a6764a5e024903848561c9935b1a43 (DEFAULT_TABLE table=default.Table_88951dd073b74c99aa4a0bd7dcf2e19d [id=51cf184bc7634141841001c082262441]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.903164 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a0691de253b4b008e5d995fd6b876c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.903371 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 9a0691de253b4b008e5d995fd6b876c2 (DEFAULT_TABLE table=default.Table_ab57b782a92747e78e2bfe28480aa0ec [id=28d43fd5660f459a9acfab2fc0dfbdb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.903878 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a0691de253b4b008e5d995fd6b876c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.906219 28651 tablet_service.cc:1511] Processing CreateTablet for tablet 69d5e75b4a8247b29d69d07fad5ca5db (DEFAULT_TABLE table=default.Table_8a78f34a376f49f88006dc80c596745f [id=5278d8eb60e4448ebae7afc3edb0db8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.906684 28520 tablet_service.cc:1511] Processing CreateTablet for tablet 37a6764a5e024903848561c9935b1a43 (DEFAULT_TABLE table=default.Table_88951dd073b74c99aa4a0bd7dcf2e19d [id=51cf184bc7634141841001c082262441]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.907509 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a6764a5e024903848561c9935b1a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.910929 28524 tablet_service.cc:1511] Processing CreateTablet for tablet ac58e939730c4af681c2f08bcfbb6602 (DEFAULT_TABLE table=default.Table_ceb33d60d86e4fd898ae67c78da94a55 [id=3251ae257e0f49b99352dcc1ef004588]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.911291 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac58e939730c4af681c2f08bcfbb6602. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.911790 28655 tablet_service.cc:1511] Processing CreateTablet for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 (DEFAULT_TABLE table=default.Table_0c3d584cea7845df83ac7adfe73d31e4 [id=38a8a1053c4c41d484c0b5bc4439dc0c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:21.912503 28227 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet edc72a06ee8e4420b07a17d2feb3fcc2 consensus state has no leader
I20260618 13:59:21.913753 28227 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 7e5517e820ae47de810ff817f5382839 (table default.Table_4fd7db2ab84d49febc9b21aa3c03d5cf [id=49576dabdecf41aca1b15ff3fc0c113c]). Expected version 2 got 0
I20260618 13:59:21.913951 28227 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 411b0c333ae7450f9ada90afee553615 (table default.Table_9241cbda69c4474e8d4af9e391d5debb [id=cd89063e7d294d3fbb3b58358e781c90]). Expected version 1 got 0
I20260618 13:59:21.914683 28525 tablet_service.cc:1511] Processing CreateTablet for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 (DEFAULT_TABLE table=default.Table_0c3d584cea7845df83ac7adfe73d31e4 [id=38a8a1053c4c41d484c0b5bc4439dc0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.914999 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.914906 28227 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_4fd7db2ab84d49febc9b21aa3c03d5cf" } new_table_name: "default.Table_454932cc31ee4f6d8515153fce3b4666" modify_external_catalogs: true
I20260618 13:59:21.917848 28522 tablet_service.cc:1511] Processing CreateTablet for tablet 69d5e75b4a8247b29d69d07fad5ca5db (DEFAULT_TABLE table=default.Table_8a78f34a376f49f88006dc80c596745f [id=5278d8eb60e4448ebae7afc3edb0db8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.918293 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69d5e75b4a8247b29d69d07fad5ca5db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.918956 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69d5e75b4a8247b29d69d07fad5ca5db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.921801 28660 tablet_service.cc:1511] Processing CreateTablet for tablet ac58e939730c4af681c2f08bcfbb6602 (DEFAULT_TABLE table=default.Table_ceb33d60d86e4fd898ae67c78da94a55 [id=3251ae257e0f49b99352dcc1ef004588]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.922299 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac58e939730c4af681c2f08bcfbb6602. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.922660 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:21.924468 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 125bb6885ab348d885d5a3b4bdd45bc7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 125bb6885ab348d885d5a3b4bdd45bc7 already in progress: creating tablet
I20260618 13:59:21.926750 28650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a6764a5e024903848561c9935b1a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.940138 28391 tablet_service.cc:1511] Processing CreateTablet for tablet ac58e939730c4af681c2f08bcfbb6602 (DEFAULT_TABLE table=default.Table_ceb33d60d86e4fd898ae67c78da94a55 [id=3251ae257e0f49b99352dcc1ef004588]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.940687 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac58e939730c4af681c2f08bcfbb6602. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.941370 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 (DEFAULT_TABLE table=default.Table_0c3d584cea7845df83ac7adfe73d31e4 [id=38a8a1053c4c41d484c0b5bc4439dc0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.941814 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.942667 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 85a3b27d914a412498919a3f65e20522 (table default.Table_94d9d315c21e4cfb9950d96262c9b6e3 [id=a874707bba8d41e7b598d7ad4a3e21f3]) successfully deleted
I20260618 13:59:21.943434 28797 tablet_bootstrap.cc:492] T eeffc96b9b97441d9301e4919d0047e1 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:21.943557 28797 ts_tablet_manager.cc:1403] T eeffc96b9b97441d9301e4919d0047e1 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.076s user 0.000s sys 0.019s
I20260618 13:59:21.944329 28797 raft_consensus.cc:359] T eeffc96b9b97441d9301e4919d0047e1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.945083 28220 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_cd35c87466e64da6bd51a0ed413b9933" } new_table_name: "default.Table_91f3283da06a48f7af7dd810eed259a3" modify_external_catalogs: true
W20260618 13:59:21.945928 28223 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8b59255b14df4158afe37cb8890e7415 consensus state has no leader
I20260618 13:59:21.946293 28223 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_5adadd2ea1314b34a29501870fec65f3" } new_table_name: "default.Table_77ced729210e46da993ea9e57db1d716" modify_external_catalogs: true
I20260618 13:59:21.946885 28221 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_e4cb8c97170c465980e9eeb3f6e4f88c" } modify_external_catalogs: true
I20260618 13:59:21.947077 28221 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_e4cb8c97170c465980e9eeb3f6e4f88c" } modify_external_catalogs: true
I20260618 13:59:21.947389 28797 raft_consensus.cc:385] T eeffc96b9b97441d9301e4919d0047e1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:21.947506 28797 raft_consensus.cc:740] T eeffc96b9b97441d9301e4919d0047e1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:21.947091 28229 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_7d3c22c138704d71a7f2a0e756cfa3a2" } modify_external_catalogs: true
I20260618 13:59:21.947950 28229 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_7d3c22c138704d71a7f2a0e756cfa3a2" } modify_external_catalogs: true
I20260618 13:59:21.948280 28797 consensus_queue.cc:260] T eeffc96b9b97441d9301e4919d0047e1 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:21.948809 28797 ts_tablet_manager.cc:1434] T eeffc96b9b97441d9301e4919d0047e1 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260618 13:59:21.953214 28228 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet ba0d05ea44e1473fb4b24f61eab096e4
I20260618 13:59:21.955273 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet ba0d05ea44e1473fb4b24f61eab096e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43976
W20260618 13:59:21.960377 28227 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7e5517e820ae47de810ff817f5382839 consensus state has no leader
W20260618 13:59:21.961354 28220 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 36b8a46a41e543c58a946b6764618c7d consensus state has no leader
I20260618 13:59:21.962404 28224 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet d9c00003300246ada2a356e3e1cf684b
W20260618 13:59:21.964031 28223 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dca54111fea047d9ae0fabe376d89382 consensus state has no leader
I20260618 13:59:21.965001 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet 85a3b27d914a412498919a3f65e20522 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.968621 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet ba0d05ea44e1473fb4b24f61eab096e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.970602 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet d9c00003300246ada2a356e3e1cf684b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.970885 28225 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet c552d1316e3340f4bcfa39618ae309b8
I20260618 13:59:21.971446 28225 catalog_manager.cc:6784] ReplaceTablet: tablet c552d1316e3340f4bcfa39618ae309b8 deleted and replaced by tablet fe750fc5d998448caeab0a6c808dfba7
I20260618 13:59:21.972651 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet d9c00003300246ada2a356e3e1cf684b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.973773 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet c552d1316e3340f4bcfa39618ae309b8 with delete_type TABLET_DATA_DELETED (replaced by tablet fe750fc5d998448caeab0a6c808dfba7) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:21.973846 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet c552d1316e3340f4bcfa39618ae309b8 with delete_type TABLET_DATA_DELETED (replaced by tablet fe750fc5d998448caeab0a6c808dfba7) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:21.973938 28382 tablet_service.cc:1511] Processing CreateTablet for tablet 69d5e75b4a8247b29d69d07fad5ca5db (DEFAULT_TABLE table=default.Table_8a78f34a376f49f88006dc80c596745f [id=5278d8eb60e4448ebae7afc3edb0db8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.974635 28382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69d5e75b4a8247b29d69d07fad5ca5db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:21.991415 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet ba0d05ea44e1473fb4b24f61eab096e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:21.991822 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet d9c00003300246ada2a356e3e1cf684b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:21.994045 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 already in progress: creating tablet
I20260618 13:59:21.994705 28651 tablet_service.cc:1511] Processing CreateTablet for tablet 9c17e76ae9b44207998a43e3576908c4 (DEFAULT_TABLE table=default.Table_c7d24bbb3b724c98ba3da942a84843e1 [id=ef849a614b5040f79ced1b5abce5313e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.995129 28798 tablet_bootstrap.cc:492] T a2ca1e22c38e49d793d413e9eaaac06f P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:21.995301 28798 ts_tablet_manager.cc:1403] T a2ca1e22c38e49d793d413e9eaaac06f P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.136s user 0.005s sys 0.000s
I20260618 13:59:21.995540 28385 tablet_service.cc:1511] Processing CreateTablet for tablet 9c17e76ae9b44207998a43e3576908c4 (DEFAULT_TABLE table=default.Table_c7d24bbb3b724c98ba3da942a84843e1 [id=ef849a614b5040f79ced1b5abce5313e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:21.995945 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c17e76ae9b44207998a43e3576908c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.001838 28802 ts_tablet_manager.cc:1929] T d6e3060118e844559803a649224d7474 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.001962 28802 log.cc:1199] T d6e3060118e844559803a649224d7474 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/d6e3060118e844559803a649224d7474
I20260618 13:59:22.002301 28802 ts_tablet_manager.cc:1950] T d6e3060118e844559803a649224d7474 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
W20260618 13:59:22.002521 28858 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
I20260618 13:59:22.003389 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet c552d1316e3340f4bcfa39618ae309b8 with delete_type TABLET_DATA_DELETED (replaced by tablet fe750fc5d998448caeab0a6c808dfba7) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.004021 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet d6e3060118e844559803a649224d7474 (table default.Table_b35e2c7c7707422d93364bf9be3bab3f [id=625859c44f3246eaacad14ba1f027d9c]) successfully deleted
W20260618 13:59:22.004247 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 8be4374ae7cf4907940ae90299bf9ba0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8be4374ae7cf4907940ae90299bf9ba0 already in progress: creating tablet
W20260618 13:59:22.004595 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet ba0d05ea44e1473fb4b24f61eab096e4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba0d05ea44e1473fb4b24f61eab096e4 already in progress: creating tablet
I20260618 13:59:22.005020 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c17e76ae9b44207998a43e3576908c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.005357 28797 tablet_bootstrap.cc:492] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:22.008240 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet d9c00003300246ada2a356e3e1cf684b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9c00003300246ada2a356e3e1cf684b already in progress: creating tablet
W20260618 13:59:22.008544 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet c552d1316e3340f4bcfa39618ae309b8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c552d1316e3340f4bcfa39618ae309b8 already in progress: creating tablet
I20260618 13:59:22.008945 28799 tablet_bootstrap.cc:492] T 31b328882af549a78535cdd55c6ea09c P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:22.009100 28799 ts_tablet_manager.cc:1403] T 31b328882af549a78535cdd55c6ea09c P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.123s user 0.051s sys 0.039s
I20260618 13:59:22.009882 28799 raft_consensus.cc:359] T 31b328882af549a78535cdd55c6ea09c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.010076 28799 raft_consensus.cc:385] T 31b328882af549a78535cdd55c6ea09c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.010161 28799 raft_consensus.cc:740] T 31b328882af549a78535cdd55c6ea09c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
W20260618 13:59:22.010218 28858 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 13:59:22.010368 28858 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260618 13:59:22.010381 28934 raft_consensus.cc:493] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:22.010589 28934 raft_consensus.cc:515] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.010921 28799 consensus_queue.cc:260] T 31b328882af549a78535cdd55c6ea09c P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.011377 28799 ts_tablet_manager.cc:1434] T 31b328882af549a78535cdd55c6ea09c P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:22.012320 28934 leader_election.cc:290] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:22.012827 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "411b0c333ae7450f9ada90afee553615" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:22.013345 28346 leader_election.cc:343] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:22.014146 28801 tablet_replica.cc:333] T 717c38fbd9cb492b9547372e2160f84f P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:22.014604 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 9c17e76ae9b44207998a43e3576908c4 (DEFAULT_TABLE table=default.Table_c7d24bbb3b724c98ba3da942a84843e1 [id=ef849a614b5040f79ced1b5abce5313e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.014644 28798 raft_consensus.cc:359] T a2ca1e22c38e49d793d413e9eaaac06f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:22.014845 28798 raft_consensus.cc:385] T a2ca1e22c38e49d793d413e9eaaac06f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260618 13:59:22.015512 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 4a01dfb4d59549399c94640f16045555 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a01dfb4d59549399c94640f16045555 already in progress: creating tablet
I20260618 13:59:22.016340 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet eeffc96b9b97441d9301e4919d0047e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.016625 28800 tablet_replica.cc:333] T eeffc96b9b97441d9301e4919d0047e1 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
W20260618 13:59:22.016633 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet ba0d05ea44e1473fb4b24f61eab096e4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba0d05ea44e1473fb4b24f61eab096e4 already in progress: creating tablet
I20260618 13:59:22.016814 28800 raft_consensus.cc:2243] T eeffc96b9b97441d9301e4919d0047e1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.016968 28800 raft_consensus.cc:2272] T eeffc96b9b97441d9301e4919d0047e1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:22.016954 28215 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 85a3b27d914a412498919a3f65e20522 because the tablet was not found. No further retry: Not found: Tablet not found: 85a3b27d914a412498919a3f65e20522
W20260618 13:59:22.017149 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d9c00003300246ada2a356e3e1cf684b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9c00003300246ada2a356e3e1cf684b already in progress: creating tablet
W20260618 13:59:22.017366 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet c552d1316e3340f4bcfa39618ae309b8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c552d1316e3340f4bcfa39618ae309b8 already in progress: creating tablet
I20260618 13:59:22.017343 28225 master_service.cc:960] ReplaceTablet: received request to replace tablet 36b8a46a41e543c58a946b6764618c7d from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:22.017776 28800 ts_tablet_manager.cc:1916] T eeffc96b9b97441d9301e4919d0047e1 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.021178 28799 tablet_bootstrap.cc:492] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:22.022748 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 41107eb1d9164a9c98060003b4027412 (DEFAULT_TABLE table=default.Table_f5abc4ffe2a848fb8805a523adf0f735 [id=3d09bbb90ecb4103b2460f7aafa2db48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.022874 28799 tablet_bootstrap.cc:654] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.023087 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41107eb1d9164a9c98060003b4027412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.025882 28797 tablet_bootstrap.cc:654] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.026961 28381 tablet_service.cc:1511] Processing CreateTablet for tablet 41107eb1d9164a9c98060003b4027412 (DEFAULT_TABLE table=default.Table_f5abc4ffe2a848fb8805a523adf0f735 [id=3d09bbb90ecb4103b2460f7aafa2db48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.027349 28381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41107eb1d9164a9c98060003b4027412. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:22.028151 28858 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 13:59:22.028230 28858 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260618 13:59:22.028301 28858 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 13:59:22.028353 28858 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260618 13:59:22.029285 28798 raft_consensus.cc:740] T a2ca1e22c38e49d793d413e9eaaac06f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.029520 28798 consensus_queue.cc:260] T a2ca1e22c38e49d793d413e9eaaac06f P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:22.030026 28798 ts_tablet_manager.cc:1434] T a2ca1e22c38e49d793d413e9eaaac06f P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.035s user 0.003s sys 0.000s
I20260618 13:59:22.030551 28391 tablet_service.cc:1511] Processing CreateTablet for tablet 316f470ea82f42fca48e4e96d49c9574 (DEFAULT_TABLE table=default.Table_f92add1c52b049f183400eb9a01c1928 [id=5415980263ed4fbea10b969e3b8facb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.030864 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 316f470ea82f42fca48e4e96d49c9574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.031198 28798 tablet_bootstrap.cc:492] T 570d36a27b7d417989ad72885d38ff63 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:22.031916 28522 tablet_service.cc:1511] Processing CreateTablet for tablet 41107eb1d9164a9c98060003b4027412 (DEFAULT_TABLE table=default.Table_f5abc4ffe2a848fb8805a523adf0f735 [id=3d09bbb90ecb4103b2460f7aafa2db48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.032286 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "411b0c333ae7450f9ada90afee553615" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:22.032580 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c17e76ae9b44207998a43e3576908c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.035090 28385 tablet_service.cc:1511] Processing CreateTablet for tablet 233040a16b0d4b61b977ca57c1afec2d (DEFAULT_TABLE table=default.Table_7e5df99a84c84d66b548875d6d934656 [id=2332167cba774a0795c11d97505049c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.035404 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 233040a16b0d4b61b977ca57c1afec2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.038882 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 8b88ed7cf79c4929a8704f66691cbed4 (DEFAULT_TABLE table=default.Table_1333d91d141a46acac33460026215195 [id=232810f09f9e41269d67ccd55527ab8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.039204 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b88ed7cf79c4929a8704f66691cbed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.042021 28389 tablet_service.cc:1511] Processing CreateTablet for tablet fddea8fba4da40d7a5780e6750caf5e9 (DEFAULT_TABLE table=default.Table_ba909970a26742b4b1006b747874b417 [id=af94ae08478c4a37b443b5f46e4dd1c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.042356 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fddea8fba4da40d7a5780e6750caf5e9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:22.050730 28346 leader_election.cc:343] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:22.050918 28346 leader_election.cc:304] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:22.052897 28382 tablet_service.cc:1511] Processing CreateTablet for tablet 092a9aba9c6e48f5be9cf5d55ba93678 (DEFAULT_TABLE table=default.Table_0a13b90182654346acbbc8f47a56436d [id=c31d65adb55f458c9ced14d598464f89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.053364 28382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 092a9aba9c6e48f5be9cf5d55ba93678. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.056514 28798 tablet_bootstrap.cc:654] T 570d36a27b7d417989ad72885d38ff63 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.065814 28934 raft_consensus.cc:2749] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:22.067269 28214 catalog_manager.cc:5100] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): alter failed for tablet 717c38fbd9cb492b9547372e2160f84f (table default.Table_6703d889c9814a9a81e4c5af4e6745f5 [id=3bcc877eddec4d00b3dc1760cc635b36]),no further retry: Not found: Tablet not found: 717c38fbd9cb492b9547372e2160f84f
I20260618 13:59:22.068629 28801 raft_consensus.cc:2243] T 717c38fbd9cb492b9547372e2160f84f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.068879 28801 raft_consensus.cc:2272] T 717c38fbd9cb492b9547372e2160f84f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.069417 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41107eb1d9164a9c98060003b4027412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.073318 28800 ts_tablet_manager.cc:1929] T eeffc96b9b97441d9301e4919d0047e1 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.073760 28800 log.cc:1199] T eeffc96b9b97441d9301e4919d0047e1 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/eeffc96b9b97441d9301e4919d0047e1
I20260618 13:59:22.073772 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 31b328882af549a78535cdd55c6ea09c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.077904 28802 tablet_replica.cc:333] T 31b328882af549a78535cdd55c6ea09c P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:22.078073 28801 ts_tablet_manager.cc:1916] T 717c38fbd9cb492b9547372e2160f84f P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.078155 28802 raft_consensus.cc:2243] T 31b328882af549a78535cdd55c6ea09c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.076023 28800 ts_tablet_manager.cc:1950] T eeffc96b9b97441d9301e4919d0047e1 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:22.078333 28802 raft_consensus.cc:2272] T 31b328882af549a78535cdd55c6ea09c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.079679 28802 ts_tablet_manager.cc:1916] T 31b328882af549a78535cdd55c6ea09c P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.081223 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 233040a16b0d4b61b977ca57c1afec2d (DEFAULT_TABLE table=default.Table_7e5df99a84c84d66b548875d6d934656 [id=2332167cba774a0795c11d97505049c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.081650 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 233040a16b0d4b61b977ca57c1afec2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.087386 28650 tablet_service.cc:1511] Processing CreateTablet for tablet 8b88ed7cf79c4929a8704f66691cbed4 (DEFAULT_TABLE table=default.Table_1333d91d141a46acac33460026215195 [id=232810f09f9e41269d67ccd55527ab8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.087419 28524 tablet_service.cc:1511] Processing CreateTablet for tablet 233040a16b0d4b61b977ca57c1afec2d (DEFAULT_TABLE table=default.Table_7e5df99a84c84d66b548875d6d934656 [id=2332167cba774a0795c11d97505049c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.087742 28650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b88ed7cf79c4929a8704f66691cbed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.087826 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 233040a16b0d4b61b977ca57c1afec2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.087833 28802 ts_tablet_manager.cc:1929] T 31b328882af549a78535cdd55c6ea09c P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.087946 28802 log.cc:1199] T 31b328882af549a78535cdd55c6ea09c P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/31b328882af549a78535cdd55c6ea09c
I20260618 13:59:22.088295 28802 ts_tablet_manager.cc:1950] T 31b328882af549a78535cdd55c6ea09c P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:22.089588 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 31b328882af549a78535cdd55c6ea09c (table default.Table_823368bbdf8242739a6022f65280f5a2 [id=28f3530b169a44eb9c262a718c8ea6d3]) successfully deleted
I20260618 13:59:22.086792 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 316f470ea82f42fca48e4e96d49c9574 (DEFAULT_TABLE table=default.Table_f92add1c52b049f183400eb9a01c1928 [id=5415980263ed4fbea10b969e3b8facb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.091497 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet eeffc96b9b97441d9301e4919d0047e1 (table default.Table_f0607bf6529e4f209e00f49908420141 [id=972bc2c124ac459da06ec4093c5b564b]) successfully deleted
I20260618 13:59:22.090891 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 316f470ea82f42fca48e4e96d49c9574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.092254 28655 tablet_service.cc:1511] Processing CreateTablet for tablet 092a9aba9c6e48f5be9cf5d55ba93678 (DEFAULT_TABLE table=default.Table_0a13b90182654346acbbc8f47a56436d [id=c31d65adb55f458c9ced14d598464f89]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:22.092432 28216 consensus_peers.cc:597] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 -> Peer a950ab7cd66b4cb0b29592aa692c2dd0 (127.27.100.61:45967): Couldn't send request to peer a950ab7cd66b4cb0b29592aa692c2dd0. Status: Network error: Client connection negotiation failed: client connection to 127.27.100.61:45967: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260618 13:59:22.091584 28520 tablet_service.cc:1511] Processing CreateTablet for tablet 092a9aba9c6e48f5be9cf5d55ba93678 (DEFAULT_TABLE table=default.Table_0a13b90182654346acbbc8f47a56436d [id=c31d65adb55f458c9ced14d598464f89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.093369 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 092a9aba9c6e48f5be9cf5d55ba93678. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.093670 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_f4a03a199e51400ca4c8a5d868ec2025"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:22.093964 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_de7bace9efe6428da6ea8e662f45c90e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.094259 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4a03a199e51400ca4c8a5d868ec2025 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:22.094466 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_de7bace9efe6428da6ea8e662f45c90e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.095839 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_68ab7758379b44628a68285f209716ef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.096278 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_68ab7758379b44628a68285f209716ef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.096652 28220 master_service.cc:960] ReplaceTablet: received request to replace tablet 8c9d89fa75174a969ded07e0596db99b from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:22.097188 28220 master_service.cc:960] ReplaceTablet: received request to replace tablet 7e5517e820ae47de810ff817f5382839 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:22.097293 28226 master_service.cc:960] ReplaceTablet: received request to replace tablet edc72a06ee8e4420b07a17d2feb3fcc2 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:22.098119 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_fdff0afd87584a53a884c9b76926d6d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.098606 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fdff0afd87584a53a884c9b76926d6d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.098990 28799 tablet_bootstrap.cc:492] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:22.099124 28799 ts_tablet_manager.cc:1403] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.078s user 0.011s sys 0.015s
I20260618 13:59:22.091715 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet a2ca1e22c38e49d793d413e9eaaac06f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.092077 28525 tablet_service.cc:1511] Processing CreateTablet for tablet 8b88ed7cf79c4929a8704f66691cbed4 (DEFAULT_TABLE table=default.Table_1333d91d141a46acac33460026215195 [id=232810f09f9e41269d67ccd55527ab8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.091857 28519 tablet_service.cc:1511] Processing CreateTablet for tablet fddea8fba4da40d7a5780e6750caf5e9 (DEFAULT_TABLE table=default.Table_ba909970a26742b4b1006b747874b417 [id=af94ae08478c4a37b443b5f46e4dd1c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.099836 28799 raft_consensus.cc:359] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.100003 28799 raft_consensus.cc:385] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.100086 28799 raft_consensus.cc:740] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.100351 28799 consensus_queue.cc:260] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.100677 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_cc20b3f971bb46969baf1fc71e217f2c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:22.100876 28799 ts_tablet_manager.cc:1434] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:22.101212 28799 tablet_bootstrap.cc:492] T 18fb8adfc8ee40e18a1da5f978317124 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:22.102226 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cc20b3f971bb46969baf1fc71e217f2c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.103309 28519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fddea8fba4da40d7a5780e6750caf5e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.103451 28799 tablet_bootstrap.cc:654] T 18fb8adfc8ee40e18a1da5f978317124 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.106866 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 092a9aba9c6e48f5be9cf5d55ba93678. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.108228 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b88ed7cf79c4929a8704f66691cbed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.109298 28221 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 5332b6bb76ea419086341738fd135b24
I20260618 13:59:22.110088 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet 5332b6bb76ea419086341738fd135b24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.110409 28802 tablet_replica.cc:333] T 5332b6bb76ea419086341738fd135b24 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:22.116916 28802 raft_consensus.cc:2243] T 5332b6bb76ea419086341738fd135b24 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.117177 28802 raft_consensus.cc:2272] T 5332b6bb76ea419086341738fd135b24 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.117743 28802 ts_tablet_manager.cc:1916] T 5332b6bb76ea419086341738fd135b24 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.118907 28799 tablet_bootstrap.cc:492] T 18fb8adfc8ee40e18a1da5f978317124 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:22.119107 28799 ts_tablet_manager.cc:1403] T 18fb8adfc8ee40e18a1da5f978317124 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.003s
I20260618 13:59:22.120627 28799 raft_consensus.cc:359] T 18fb8adfc8ee40e18a1da5f978317124 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.120847 28799 raft_consensus.cc:385] T 18fb8adfc8ee40e18a1da5f978317124 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.120937 28799 raft_consensus.cc:740] T 18fb8adfc8ee40e18a1da5f978317124 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.121224 28799 consensus_queue.cc:260] T 18fb8adfc8ee40e18a1da5f978317124 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.122139 28799 ts_tablet_manager.cc:1434] T 18fb8adfc8ee40e18a1da5f978317124 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260618 13:59:22.122483 28656 tablet_service.cc:1511] Processing CreateTablet for tablet fddea8fba4da40d7a5780e6750caf5e9 (DEFAULT_TABLE table=default.Table_ba909970a26742b4b1006b747874b417 [id=af94ae08478c4a37b443b5f46e4dd1c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.124766 28799 tablet_bootstrap.cc:492] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:22.126924 28810 raft_consensus.cc:493] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:22.128031 28802 ts_tablet_manager.cc:1929] T 5332b6bb76ea419086341738fd135b24 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.128149 28802 log.cc:1199] T 5332b6bb76ea419086341738fd135b24 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/5332b6bb76ea419086341738fd135b24
I20260618 13:59:22.128605 28934 raft_consensus.cc:493] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:22.128609 28799 tablet_bootstrap.cc:654] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.128743 28934 raft_consensus.cc:515] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:22.129395 28934 leader_election.cc:290] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:22.129876 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "570d36a27b7d417989ad72885d38ff63" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:22.130013 28802 ts_tablet_manager.cc:1950] T 5332b6bb76ea419086341738fd135b24 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:22.127143 28810 raft_consensus.cc:515] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:22.130632 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fddea8fba4da40d7a5780e6750caf5e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.130656 28683 raft_consensus.cc:2468] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:22.130966 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 5332b6bb76ea419086341738fd135b24 (table default.Table_e4cb8c97170c465980e9eeb3f6e4f88c [id=116d7007e8754aab8d37956bcb596f13]) successfully deleted
I20260618 13:59:22.131263 28346 leader_election.cc:304] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:22.131458 28651 tablet_service.cc:1511] Processing CreateTablet for tablet 316f470ea82f42fca48e4e96d49c9574 (DEFAULT_TABLE table=default.Table_f92add1c52b049f183400eb9a01c1928 [id=5415980263ed4fbea10b969e3b8facb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.131896 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 316f470ea82f42fca48e4e96d49c9574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.132443 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "570d36a27b7d417989ad72885d38ff63" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:22.132946 28810 leader_election.cc:290] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:22.138859 28934 raft_consensus.cc:2804] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:22.138998 28934 raft_consensus.cc:493] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:22.139103 28934 raft_consensus.cc:3060] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:22.142571 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 5332b6bb76ea419086341738fd135b24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.144912 28934 raft_consensus.cc:515] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:22.145304 28797 tablet_bootstrap.cc:492] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:22.145466 28797 ts_tablet_manager.cc:1403] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.140s user 0.014s sys 0.029s
I20260618 13:59:22.145682 28934 leader_election.cc:290] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:22.145974 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "570d36a27b7d417989ad72885d38ff63" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:22.146162 28683 raft_consensus.cc:3060] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:22.149017 28683 raft_consensus.cc:2468] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:22.149233 28798 tablet_bootstrap.cc:492] T 570d36a27b7d417989ad72885d38ff63 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:22.149391 28798 ts_tablet_manager.cc:1403] T 570d36a27b7d417989ad72885d38ff63 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.118s user 0.002s sys 0.001s
I20260618 13:59:22.149369 28346 leader_election.cc:304] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:22.149617 28801 ts_tablet_manager.cc:1929] T 717c38fbd9cb492b9547372e2160f84f P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.149757 28801 log.cc:1199] T 717c38fbd9cb492b9547372e2160f84f P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/717c38fbd9cb492b9547372e2160f84f
I20260618 13:59:22.152458 28934 raft_consensus.cc:2804] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:22.153290 28799 tablet_bootstrap.cc:492] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:22.153522 28799 ts_tablet_manager.cc:1403] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.029s user 0.000s sys 0.005s
I20260618 13:59:22.154170 28858 master_runner.cc:387] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.100.20:36797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.27.100.62:43001,127.27.100.61:45967,127.27.100.60:42083
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.100.61:45967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.100.61
--webserver_port=35663
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision faec34f53d4c9dd1e3a6dd2b7314d2fc7aadff6c
build type FASTDEBUG
built by None at 18 Jun 2026 13:43:31 UTC on bdcb31816ec0
build id 12454
ASAN enabled
I20260618 13:59:22.154327 28799 raft_consensus.cc:359] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.152680 28934 raft_consensus.cc:697] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:22.154927 28799 raft_consensus.cc:385] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.155041 28799 raft_consensus.cc:740] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.155229 28934 consensus_queue.cc:237] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:22.155298 28799 consensus_queue.cc:260] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.155500 28807 raft_consensus.cc:493] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:22.155648 28807 raft_consensus.cc:515] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.156359 28807 leader_election.cc:290] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:22.156599 28858 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 13:59:22.162360 28799 ts_tablet_manager.cc:1434] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.009s user 0.004s sys 0.004s
I20260618 13:59:22.162703 28799 tablet_bootstrap.cc:492] T ebf502f25e9b4342bfaa58f42db7f2f5 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:22.164245 28229 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 570d36a27b7d417989ad72885d38ff63
I20260618 13:59:22.164979 28229 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_7e0256d5218f469c9fef22ba76bb4e64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.165537 28229 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e0256d5218f469c9fef22ba76bb4e64 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.165549 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet 570d36a27b7d417989ad72885d38ff63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.166193 28802 tablet_replica.cc:333] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:22.166393 28802 raft_consensus.cc:2243] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:22.166678 28802 pending_rounds.cc:70] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b: Trying to abort 1 pending ops.
I20260618 13:59:22.166903 28224 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_c1c5efdc5332431fbb8eef6bd34961eb" } modify_external_catalogs: true
I20260618 13:59:22.166806 28798 raft_consensus.cc:359] T 570d36a27b7d417989ad72885d38ff63 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:22.167098 28224 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_c1c5efdc5332431fbb8eef6bd34961eb" } modify_external_catalogs: true
I20260618 13:59:22.167136 28798 raft_consensus.cc:385] T 570d36a27b7d417989ad72885d38ff63 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.167215 28798 raft_consensus.cc:740] T 570d36a27b7d417989ad72885d38ff63 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.167213 28802 pending_rounds.cc:77] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298216600198500352 op_type: NO_OP noop_request { }
I20260618 13:59:22.167325 28802 raft_consensus.cc:2889] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:22.167438 28802 raft_consensus.cc:2272] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.168244 28798 consensus_queue.cc:260] T 570d36a27b7d417989ad72885d38ff63 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:22.168782 28798 ts_tablet_manager.cc:1434] T 570d36a27b7d417989ad72885d38ff63 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.019s user 0.001s sys 0.001s
I20260618 13:59:22.169147 28798 tablet_bootstrap.cc:492] T 102a500268574e428cb2d45e52849f57 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:22.170881 28798 tablet_bootstrap.cc:654] T 102a500268574e428cb2d45e52849f57 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:22.168808 28947 consensus_queue.cc:1175] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6fc808af0e884ed8836561665d8e0e7b" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260618 13:59:22.173995 28802 ts_tablet_manager.cc:1916] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.181448 28222 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_f92add1c52b049f183400eb9a01c1928" } new_table_name: "default.Table_28c746533f45405791fc12912e8fbcd4" modify_external_catalogs: true
I20260618 13:59:22.182340 28799 tablet_bootstrap.cc:654] T ebf502f25e9b4342bfaa58f42db7f2f5 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.183089 28801 ts_tablet_manager.cc:1950] T 717c38fbd9cb492b9547372e2160f84f P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:22.183280 28228 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_6703d889c9814a9a81e4c5af4e6745f5" } new_table_name: "default.Table_9b2c93a93eb1461994ae0ff79108098b" modify_external_catalogs: true
I20260618 13:59:22.184413 28225 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 36b8a46a41e543c58a946b6764618c7d
I20260618 13:59:22.184430 28801 tablet_replica.cc:333] T ee783d2c7e274158b3de2cc186f50903 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:22.184646 28801 raft_consensus.cc:2243] T ee783d2c7e274158b3de2cc186f50903 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.184855 28225 catalog_manager.cc:6784] ReplaceTablet: tablet 36b8a46a41e543c58a946b6764618c7d deleted and replaced by tablet f89c9954d48e407c96c294c26644e25c
I20260618 13:59:22.184872 28801 raft_consensus.cc:2272] T ee783d2c7e274158b3de2cc186f50903 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.185155 28225 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_b0e3e2341f764392af42b949d03dd1f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.185640 28225 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b0e3e2341f764392af42b949d03dd1f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.186789 28220 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 7e5517e820ae47de810ff817f5382839
I20260618 13:59:22.187197 28220 catalog_manager.cc:6784] ReplaceTablet: tablet 7e5517e820ae47de810ff817f5382839 deleted and replaced by tablet dc36fbea853a4d4e9f61909200239aed
I20260618 13:59:22.187455 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_74620afb05b24bf9912f3984efd632aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.187892 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_74620afb05b24bf9912f3984efd632aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.188964 28226 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet edc72a06ee8e4420b07a17d2feb3fcc2
I20260618 13:59:22.189384 28226 catalog_manager.cc:6784] ReplaceTablet: tablet edc72a06ee8e4420b07a17d2feb3fcc2 deleted and replaced by tablet aa8cf927bed9437086efd8e0f21ba26a
I20260618 13:59:22.189661 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_061bec38727b4ee69096fe3b50296f94"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.190060 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_061bec38727b4ee69096fe3b50296f94 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.191517 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_e601511ed59d407b86040479045e84ec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.191992 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e601511ed59d407b86040479045e84ec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.194664 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet 36b8a46a41e543c58a946b6764618c7d with delete_type TABLET_DATA_DELETED (replaced by tablet f89c9954d48e407c96c294c26644e25c) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.194778 28799 tablet_bootstrap.cc:492] T ebf502f25e9b4342bfaa58f42db7f2f5 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:22.194942 28799 ts_tablet_manager.cc:1403] T ebf502f25e9b4342bfaa58f42db7f2f5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.005s
I20260618 13:59:22.194972 28382 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e5517e820ae47de810ff817f5382839 with delete_type TABLET_DATA_DELETED (replaced by tablet dc36fbea853a4d4e9f61909200239aed) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.195195 28381 tablet_service.cc:1558] Processing DeleteTablet for tablet edc72a06ee8e4420b07a17d2feb3fcc2 with delete_type TABLET_DATA_DELETED (replaced by tablet aa8cf927bed9437086efd8e0f21ba26a) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.195700 28799 raft_consensus.cc:359] T ebf502f25e9b4342bfaa58f42db7f2f5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.195880 28799 raft_consensus.cc:385] T ebf502f25e9b4342bfaa58f42db7f2f5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.196362 28799 raft_consensus.cc:740] T ebf502f25e9b4342bfaa58f42db7f2f5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.196607 28799 consensus_queue.cc:260] T ebf502f25e9b4342bfaa58f42db7f2f5 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.197211 28802 ts_tablet_manager.cc:1929] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:22.197391 28802 log.cc:1199] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/570d36a27b7d417989ad72885d38ff63
I20260618 13:59:22.197947 28802 ts_tablet_manager.cc:1950] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:22.198688 28797 raft_consensus.cc:359] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.199064 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 570d36a27b7d417989ad72885d38ff63 (table default.Table_7d3c22c138704d71a7f2a0e756cfa3a2 [id=e38a333ca62d42518ab53a9ac17df2b7]) successfully deleted
I20260618 13:59:22.199549 28223 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_ceb33d60d86e4fd898ae67c78da94a55" } modify_external_catalogs: true
I20260618 13:59:22.199625 28799 ts_tablet_manager.cc:1434] T ebf502f25e9b4342bfaa58f42db7f2f5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260618 13:59:22.199949 28799 tablet_bootstrap.cc:492] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:22.199927 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet 5332b6bb76ea419086341738fd135b24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.199986 28223 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_ceb33d60d86e4fd898ae67c78da94a55" } modify_external_catalogs: true
I20260618 13:59:22.200212 28797 raft_consensus.cc:385] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.200317 28797 raft_consensus.cc:740] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.200618 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ebed81a21b14db48ecb69299e0956cd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:22.201056 28797 consensus_queue.cc:260] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:22.201283 28346 leader_election.cc:343] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:22.201658 28797 ts_tablet_manager.cc:1434] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.056s user 0.001s sys 0.001s
I20260618 13:59:22.202992 28799 tablet_bootstrap.cc:654] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.208122 28195 master_runner.cc:187] Error getting master registration for 127.27.100.61:45967: Network error: Client connection negotiation failed: client connection to 127.27.100.61:45967: connect: Connection refused (error 111),
I20260618 13:59:22.208604 28195 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260618 13:59:22.210117 28214 catalog_manager.cc:5100] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): alter failed for tablet 717c38fbd9cb492b9547372e2160f84f (table default.Table_6703d889c9814a9a81e4c5af4e6745f5 [id=3bcc877eddec4d00b3dc1760cc635b36]),no further retry: Not found: Tablet not found: 717c38fbd9cb492b9547372e2160f84f
W20260618 13:59:22.210306 28484 leader_election.cc:343] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 144aa8984a44410fb3f90dc99e9b2274
I20260618 13:59:22.210883 28797 tablet_bootstrap.cc:492] T 7e5517e820ae47de810ff817f5382839 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:22.212764 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 570d36a27b7d417989ad72885d38ff63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.213510 28800 tablet_replica.cc:333] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:22.213726 28800 raft_consensus.cc:2243] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.213941 28800 raft_consensus.cc:2272] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.214380 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet 36b8a46a41e543c58a946b6764618c7d with delete_type TABLET_DATA_DELETED (replaced by tablet f89c9954d48e407c96c294c26644e25c) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.214743 28797 tablet_bootstrap.cc:654] T 7e5517e820ae47de810ff817f5382839 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.214753 28800 ts_tablet_manager.cc:1916] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.215658 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e5517e820ae47de810ff817f5382839 with delete_type TABLET_DATA_DELETED (replaced by tablet dc36fbea853a4d4e9f61909200239aed) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.217850 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet edc72a06ee8e4420b07a17d2feb3fcc2 with delete_type TABLET_DATA_DELETED (replaced by tablet aa8cf927bed9437086efd8e0f21ba26a) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.219247 28799 tablet_bootstrap.cc:492] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:22.219419 28799 ts_tablet_manager.cc:1403] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.001s
W20260618 13:59:22.219462 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 36b8a46a41e543c58a946b6764618c7d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36b8a46a41e543c58a946b6764618c7d already in progress: creating tablet
I20260618 13:59:22.219812 28802 tablet_replica.cc:333] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:22.220175 28802 raft_consensus.cc:2243] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.220363 28802 raft_consensus.cc:2272] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.220294 28799 raft_consensus.cc:359] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.220505 28799 raft_consensus.cc:385] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.220602 28799 raft_consensus.cc:740] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.221100 28802 ts_tablet_manager.cc:1916] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:22.221619 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 5332b6bb76ea419086341738fd135b24 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5332b6bb76ea419086341738fd135b24 already in progress: creating tablet
I20260618 13:59:22.225237 28800 ts_tablet_manager.cc:1929] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.225406 28800 log.cc:1199] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/570d36a27b7d417989ad72885d38ff63
I20260618 13:59:22.225852 28800 ts_tablet_manager.cc:1950] T 570d36a27b7d417989ad72885d38ff63 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:22.226176 28797 tablet_bootstrap.cc:492] T 7e5517e820ae47de810ff817f5382839 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:22.226363 28797 ts_tablet_manager.cc:1403] T 7e5517e820ae47de810ff817f5382839 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.002s
I20260618 13:59:22.226965 28799 consensus_queue.cc:260] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.227314 28797 raft_consensus.cc:359] T 7e5517e820ae47de810ff817f5382839 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.227541 28797 raft_consensus.cc:385] T 7e5517e820ae47de810ff817f5382839 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.227869 28797 raft_consensus.cc:740] T 7e5517e820ae47de810ff817f5382839 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.228012 28799 ts_tablet_manager.cc:1434] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20260618 13:59:22.228204 28797 consensus_queue.cc:260] T 7e5517e820ae47de810ff817f5382839 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.228749 28797 ts_tablet_manager.cc:1434] T 7e5517e820ae47de810ff817f5382839 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 13:59:22.229092 28797 tablet_bootstrap.cc:492] T d6e3060118e844559803a649224d7474 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:22.230994 28797 tablet_bootstrap.cc:654] T d6e3060118e844559803a649224d7474 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.231786 28799 tablet_bootstrap.cc:492] T d980cda1c60e4edca7b4d7537f4f5837 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:22.232776 28346 leader_election.cc:343] T 570d36a27b7d417989ad72885d38ff63 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:22.232965 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 570d36a27b7d417989ad72885d38ff63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.233628 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet 36b8a46a41e543c58a946b6764618c7d with delete_type TABLET_DATA_DELETED (replaced by tablet f89c9954d48e407c96c294c26644e25c) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.234059 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e5517e820ae47de810ff817f5382839 with delete_type TABLET_DATA_DELETED (replaced by tablet dc36fbea853a4d4e9f61909200239aed) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.234490 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet edc72a06ee8e4420b07a17d2feb3fcc2 with delete_type TABLET_DATA_DELETED (replaced by tablet aa8cf927bed9437086efd8e0f21ba26a) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.235023 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "570d36a27b7d417989ad72885d38ff63" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:22.235224 28549 raft_consensus.cc:3060] T 570d36a27b7d417989ad72885d38ff63 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:22.235589 28801 ts_tablet_manager.cc:1916] T ee783d2c7e274158b3de2cc186f50903 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.236080 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 717c38fbd9cb492b9547372e2160f84f (table default.Table_6703d889c9814a9a81e4c5af4e6745f5 [id=3bcc877eddec4d00b3dc1760cc635b36]) successfully deleted
I20260618 13:59:22.236281 28799 tablet_bootstrap.cc:654] T d980cda1c60e4edca7b4d7537f4f5837 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:22.236315 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 0a51e830e3014726b1d1c2608973f036 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a51e830e3014726b1d1c2608973f036 already in progress: creating tablet
I20260618 13:59:22.236672 28800 tablet_replica.cc:333] T 7e5517e820ae47de810ff817f5382839 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:22.236877 28800 raft_consensus.cc:2243] T 7e5517e820ae47de810ff817f5382839 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.237043 28800 raft_consensus.cc:2272] T 7e5517e820ae47de810ff817f5382839 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.237775 28802 ts_tablet_manager.cc:1929] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.237967 28802 log.cc:1199] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/7e5517e820ae47de810ff817f5382839
I20260618 13:59:22.238370 28802 ts_tablet_manager.cc:1950] T 7e5517e820ae47de810ff817f5382839 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:22.239306 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 7e5517e820ae47de810ff817f5382839 (table default.Table_454932cc31ee4f6d8515153fce3b4666 [id=49576dabdecf41aca1b15ff3fc0c113c]) successfully deleted
W20260618 13:59:22.239985 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet edc72a06ee8e4420b07a17d2feb3fcc2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet edc72a06ee8e4420b07a17d2feb3fcc2 already in progress: creating tablet
I20260618 13:59:22.240217 28056 master_runner.cc:187] Error getting master registration for 127.27.100.61:45967: Network error: Client connection negotiation failed: client connection to 127.27.100.61:45967: connect: Connection refused (error 111),
I20260618 13:59:22.240356 28056 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260618 13:59:22.241953 28224 catalog_manager.cc:5978] Not sending DeleteTablet requests; no consensus state for tablet 4305c31719d44a83aa0bbe6f47180aa8
I20260618 13:59:22.242170 28800 ts_tablet_manager.cc:1916] T 7e5517e820ae47de810ff817f5382839 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:22.242631 28228 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8b59255b14df4158afe37cb8890e7415 consensus state has no leader
W20260618 13:59:22.243855 28483 leader_election.cc:343] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 144aa8984a44410fb3f90dc99e9b2274
I20260618 13:59:22.243991 28483 leader_election.cc:304] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:22.244650 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ebed81a21b14db48ecb69299e0956cd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:22.244719 28228 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_0827d01c77434679a1d04c782d175304" } new_table_name: "default.Table_fe8f6240599e467a92887643cc9d210b" modify_external_catalogs: true
I20260618 13:59:22.245329 28224 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_91f3283da06a48f7af7dd810eed259a3" } new_table_name: "default.Table_e6090de33b2147489b3af52d6ec4dc1b" modify_external_catalogs: true
I20260618 13:59:22.245548 28220 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_0c3d584cea7845df83ac7adfe73d31e4" } new_table_name: "default.Table_e372385b751446e28fe8798e387b2148" modify_external_catalogs: true
I20260618 13:59:22.246557 28225 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_88951dd073b74c99aa4a0bd7dcf2e19d" } modify_external_catalogs: true
I20260618 13:59:22.247243 28225 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_88951dd073b74c99aa4a0bd7dcf2e19d" } modify_external_catalogs: true
I20260618 13:59:22.250367 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 570d36a27b7d417989ad72885d38ff63 (table default.Table_7d3c22c138704d71a7f2a0e756cfa3a2 [id=e38a333ca62d42518ab53a9ac17df2b7]) successfully deleted
W20260618 13:59:22.251499 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 36b8a46a41e543c58a946b6764618c7d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36b8a46a41e543c58a946b6764618c7d already in progress: creating tablet
I20260618 13:59:22.253271 28797 tablet_bootstrap.cc:492] T d6e3060118e844559803a649224d7474 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:22.253516 28797 ts_tablet_manager.cc:1403] T d6e3060118e844559803a649224d7474 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.001s
I20260618 13:59:22.254179 28799 tablet_bootstrap.cc:492] T d980cda1c60e4edca7b4d7537f4f5837 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:22.254361 28799 ts_tablet_manager.cc:1403] T d980cda1c60e4edca7b4d7537f4f5837 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.002s
I20260618 13:59:22.254534 28797 raft_consensus.cc:359] T d6e3060118e844559803a649224d7474 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.254798 28797 raft_consensus.cc:385] T d6e3060118e844559803a649224d7474 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.254928 28797 raft_consensus.cc:740] T d6e3060118e844559803a649224d7474 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.255111 28799 raft_consensus.cc:359] T d980cda1c60e4edca7b4d7537f4f5837 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.255318 28799 raft_consensus.cc:385] T d980cda1c60e4edca7b4d7537f4f5837 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.255398 28799 raft_consensus.cc:740] T d980cda1c60e4edca7b4d7537f4f5837 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.255228 28797 consensus_queue.cc:260] T d6e3060118e844559803a649224d7474 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.255904 28797 ts_tablet_manager.cc:1434] T d6e3060118e844559803a649224d7474 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:22.256316 28797 tablet_bootstrap.cc:492] T 059716cec7f94ea6ad8989fecc6f832a P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:22.256285 28858 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20260618 13:59:22.256942 28799 consensus_queue.cc:260] T d980cda1c60e4edca7b4d7537f4f5837 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.261158 28799 ts_tablet_manager.cc:1434] T d980cda1c60e4edca7b4d7537f4f5837 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260618 13:59:22.262313 28799 tablet_bootstrap.cc:492] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:22.266048 28226 master_service.cc:960] ReplaceTablet: received request to replace tablet ac58e939730c4af681c2f08bcfbb6602 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:22.266906 28223 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet ac58e939730c4af681c2f08bcfbb6602
I20260618 13:59:22.267709 28799 tablet_bootstrap.cc:654] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.268422 28381 tablet_service.cc:1558] Processing DeleteTablet for tablet ac58e939730c4af681c2f08bcfbb6602 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:22.269119 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet ac58e939730c4af681c2f08bcfbb6602 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ac58e939730c4af681c2f08bcfbb6602 already in progress: creating tablet
I20260618 13:59:22.274993 28651 tablet_service.cc:1558] Processing DeleteTablet for tablet ac58e939730c4af681c2f08bcfbb6602 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43976
W20260618 13:59:22.275034 28216 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet eeffc96b9b97441d9301e4919d0047e1 (table default.Table_f0607bf6529e4f209e00f49908420141 [id=972bc2c124ac459da06ec4093c5b564b], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet eeffc96b9b97441d9301e4919d0047e1 consensus state has no leader
I20260618 13:59:22.277304 28799 tablet_bootstrap.cc:492] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:22.277451 28799 ts_tablet_manager.cc:1403] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.000s
I20260618 13:59:22.278203 28799 raft_consensus.cc:359] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.278368 28799 raft_consensus.cc:385] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.278477 28799 raft_consensus.cc:740] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.278690 28799 consensus_queue.cc:260] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.282732 28799 ts_tablet_manager.cc:1434] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260618 13:59:22.283524 28797 tablet_bootstrap.cc:654] T 059716cec7f94ea6ad8989fecc6f832a P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.285210 28799 tablet_bootstrap.cc:492] T 1f0109536c24485c9934193f2ef8aeae P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:22.286840 28224 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f89c9954d48e407c96c294c26644e25c has no consensus state
W20260618 13:59:22.287191 28220 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 consensus state has no leader
W20260618 13:59:22.287837 28222 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 316f470ea82f42fca48e4e96d49c9574 consensus state has no leader
I20260618 13:59:22.290024 28799 tablet_bootstrap.cc:654] T 1f0109536c24485c9934193f2ef8aeae P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.299048 28799 tablet_bootstrap.cc:492] T 1f0109536c24485c9934193f2ef8aeae P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
W20260618 13:59:22.300558 28228 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 864f2602840c40fc8606b3784e671cba consensus state has no leader
I20260618 13:59:22.299669 28799 ts_tablet_manager.cc:1403] T 1f0109536c24485c9934193f2ef8aeae P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.000s
I20260618 13:59:22.303126 28800 ts_tablet_manager.cc:1929] T 7e5517e820ae47de810ff817f5382839 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.303305 28800 log.cc:1199] T 7e5517e820ae47de810ff817f5382839 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/7e5517e820ae47de810ff817f5382839
I20260618 13:59:22.303715 28800 ts_tablet_manager.cc:1950] T 7e5517e820ae47de810ff817f5382839 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:22.303062 28799 raft_consensus.cc:359] T 1f0109536c24485c9934193f2ef8aeae P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.304051 28799 raft_consensus.cc:385] T 1f0109536c24485c9934193f2ef8aeae P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.304308 28799 raft_consensus.cc:740] T 1f0109536c24485c9934193f2ef8aeae P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.304752 28799 consensus_queue.cc:260] T 1f0109536c24485c9934193f2ef8aeae P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.307986 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 7e5517e820ae47de810ff817f5382839 (table default.Table_454932cc31ee4f6d8515153fce3b4666 [id=49576dabdecf41aca1b15ff3fc0c113c]) successfully deleted
I20260618 13:59:22.308562 28229 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 29bf263d20934ed6a6efb4b215252c00 (table default.Table_5f8fbc1f56c1484cb99b55730190bbcf [id=7923680bde144e3dbb78c922d09540b9]). Expected version 4 got 0
I20260618 13:59:22.309721 28221 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 311ba4dd83984f379c8f4f8274bc26c1 (table default.Table_dbc49898f77e441cb501871dfae61db4 [id=d8ef984c664f4a6598c70a48e9aff826]). Expected version 2 got 0
W20260618 13:59:22.309149 28483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 71414.2us
I20260618 13:59:22.315359 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet ac58e939730c4af681c2f08bcfbb6602 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.317672 28221 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 311ba4dd83984f379c8f4f8274bc26c1 (table default.Table_dbc49898f77e441cb501871dfae61db4 [id=d8ef984c664f4a6598c70a48e9aff826]). Expected version 2 got 0
I20260618 13:59:22.317683 28549 raft_consensus.cc:2468] T 570d36a27b7d417989ad72885d38ff63 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:22.318581 28221 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 37fedd27109944e2b3c870418c117758 (table default.Table_e9f01ff23601409189a5923e3755f4f8 [id=97b635f1072a4d82a82753d618600d4a]). Expected version 1 got 0
I20260618 13:59:22.318775 28798 tablet_bootstrap.cc:492] T 102a500268574e428cb2d45e52849f57 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:22.318933 28798 ts_tablet_manager.cc:1403] T 102a500268574e428cb2d45e52849f57 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.150s user 0.006s sys 0.029s
W20260618 13:59:22.318917 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet edc72a06ee8e4420b07a17d2feb3fcc2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet edc72a06ee8e4420b07a17d2feb3fcc2 already in progress: creating tablet
I20260618 13:59:22.318969 28221 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 4ebed81a21b14db48ecb69299e0956cd (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
W20260618 13:59:22.319196 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet ac58e939730c4af681c2f08bcfbb6602 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ac58e939730c4af681c2f08bcfbb6602 already in progress: creating tablet
I20260618 13:59:22.319795 28798 raft_consensus.cc:359] T 102a500268574e428cb2d45e52849f57 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:22.320041 28798 raft_consensus.cc:385] T 102a500268574e428cb2d45e52849f57 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.320164 28798 raft_consensus.cc:740] T 102a500268574e428cb2d45e52849f57 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.320436 28798 consensus_queue.cc:260] T 102a500268574e428cb2d45e52849f57 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:22.321237 28651 tablet_service.cc:1511] Processing CreateTablet for tablet dd2385e6752d49429422fc854539e9c6 (DEFAULT_TABLE table=default.Table_dc44ae6b83d844b6a16ce10ba6dce0ad [id=2e7a199c805d4453b2137b5fa69d0b4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.321652 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd2385e6752d49429422fc854539e9c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.324427 28810 raft_consensus.cc:2749] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:22.327497 28798 ts_tablet_manager.cc:1434] T 102a500268574e428cb2d45e52849f57 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.008s user 0.001s sys 0.006s
I20260618 13:59:22.327922 28798 tablet_bootstrap.cc:492] T d5cb3137659a4481840fe04174ef682d P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:22.330603 28798 tablet_bootstrap.cc:654] T d5cb3137659a4481840fe04174ef682d P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:22.333369 28960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 13:59:22.333982 28651 tablet_service.cc:1558] Processing DeleteTablet for tablet d6e3060118e844559803a649224d7474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
W20260618 13:59:22.334187 28966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 13:59:22.334555 28800 tablet_replica.cc:333] T d6e3060118e844559803a649224d7474 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:22.334769 28800 raft_consensus.cc:2243] T d6e3060118e844559803a649224d7474 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.334954 28800 raft_consensus.cc:2272] T d6e3060118e844559803a649224d7474 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.335019 28520 tablet_service.cc:1511] Processing CreateTablet for tablet dd2385e6752d49429422fc854539e9c6 (DEFAULT_TABLE table=default.Table_dc44ae6b83d844b6a16ce10ba6dce0ad [id=2e7a199c805d4453b2137b5fa69d0b4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.335176 28651 tablet_service.cc:1511] Processing CreateTablet for tablet d502d67826ff4d7a8b0f515e5d8b6d77 (DEFAULT_TABLE table=default.Table_b5800aca80594ec1bd3a86ec47dbfc64 [id=2710734517a24ad6baf0d47bf23e9bdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.335402 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd2385e6752d49429422fc854539e9c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.335526 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d502d67826ff4d7a8b0f515e5d8b6d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.336289 28800 ts_tablet_manager.cc:1916] T d6e3060118e844559803a649224d7474 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.339740 28522 tablet_service.cc:1511] Processing CreateTablet for tablet d502d67826ff4d7a8b0f515e5d8b6d77 (DEFAULT_TABLE table=default.Table_b5800aca80594ec1bd3a86ec47dbfc64 [id=2710734517a24ad6baf0d47bf23e9bdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.339813 28655 tablet_service.cc:1511] Processing CreateTablet for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e (DEFAULT_TABLE table=default.Table_45297e5c8f754fbbb3ab4dac6de1e136 [id=8c16c31ead484af3a1f3ee482b4cc233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.340119 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d502d67826ff4d7a8b0f515e5d8b6d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.340137 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.313851 28381 tablet_service.cc:1511] Processing CreateTablet for tablet dd2385e6752d49429422fc854539e9c6 (DEFAULT_TABLE table=default.Table_dc44ae6b83d844b6a16ce10ba6dce0ad [id=2e7a199c805d4453b2137b5fa69d0b4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.342314 28381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd2385e6752d49429422fc854539e9c6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:22.314594 28346 leader_election.cc:343] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:22.346149 28346 leader_election.cc:304] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:22.346791 28520 tablet_service.cc:1511] Processing CreateTablet for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e (DEFAULT_TABLE table=default.Table_45297e5c8f754fbbb3ab4dac6de1e136 [id=8c16c31ead484af3a1f3ee482b4cc233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.346903 28655 tablet_service.cc:1511] Processing CreateTablet for tablet fe750fc5d998448caeab0a6c808dfba7 (DEFAULT_TABLE table=default.Table_be74477d3cd94a6891d879db47d3efd1 [id=aec77306ae6d47ec91b286291c3ab664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.347115 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.347224 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe750fc5d998448caeab0a6c808dfba7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.347417 28388 tablet_service.cc:1558] Processing DeleteTablet for tablet ebf502f25e9b4342bfaa58f42db7f2f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.347676 28802 tablet_replica.cc:333] T ebf502f25e9b4342bfaa58f42db7f2f5 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:22.347860 28802 raft_consensus.cc:2243] T ebf502f25e9b4342bfaa58f42db7f2f5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.348037 28802 raft_consensus.cc:2272] T ebf502f25e9b4342bfaa58f42db7f2f5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.348181 28519 tablet_service.cc:1511] Processing CreateTablet for tablet fe750fc5d998448caeab0a6c808dfba7 (DEFAULT_TABLE table=default.Table_be74477d3cd94a6891d879db47d3efd1 [id=aec77306ae6d47ec91b286291c3ab664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.348263 28382 tablet_service.cc:1511] Processing CreateTablet for tablet d502d67826ff4d7a8b0f515e5d8b6d77 (DEFAULT_TABLE table=default.Table_b5800aca80594ec1bd3a86ec47dbfc64 [id=2710734517a24ad6baf0d47bf23e9bdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.348368 28946 raft_consensus.cc:2749] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:22.348520 28519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe750fc5d998448caeab0a6c808dfba7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.348627 28382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d502d67826ff4d7a8b0f515e5d8b6d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.348991 28802 ts_tablet_manager.cc:1916] T ebf502f25e9b4342bfaa58f42db7f2f5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.350737 28385 tablet_service.cc:1511] Processing CreateTablet for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e (DEFAULT_TABLE table=default.Table_45297e5c8f754fbbb3ab4dac6de1e136 [id=8c16c31ead484af3a1f3ee482b4cc233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.351081 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.354233 28391 tablet_service.cc:1511] Processing CreateTablet for tablet fe750fc5d998448caeab0a6c808dfba7 (DEFAULT_TABLE table=default.Table_be74477d3cd94a6891d879db47d3efd1 [id=aec77306ae6d47ec91b286291c3ab664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.354596 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe750fc5d998448caeab0a6c808dfba7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.355756 28802 ts_tablet_manager.cc:1929] T ebf502f25e9b4342bfaa58f42db7f2f5 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.355878 28802 log.cc:1199] T ebf502f25e9b4342bfaa58f42db7f2f5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/ebf502f25e9b4342bfaa58f42db7f2f5
I20260618 13:59:22.356262 28802 ts_tablet_manager.cc:1950] T ebf502f25e9b4342bfaa58f42db7f2f5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:22.357287 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet ebf502f25e9b4342bfaa58f42db7f2f5 (table default.Table_c1cda1e770c24c7eb9ab6c1ad8a7e786 [id=19c30a68cf504a179bbb2a966b68445c]) successfully deleted
I20260618 13:59:22.357836 28221 master_service.cc:960] ReplaceTablet: received request to replace tablet d980cda1c60e4edca7b4d7537f4f5837 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:22.358563 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 18fb8adfc8ee40e18a1da5f978317124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.358777 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet ffd7d5ebaa94416a8eaf3e61bed945e1 with delete_type TABLET_DATA_DELETED (replaced by tablet d6871232860b41d489e34d95eb897241) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.360177 28798 tablet_bootstrap.cc:492] T d5cb3137659a4481840fe04174ef682d P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:22.360379 28798 ts_tablet_manager.cc:1403] T d5cb3137659a4481840fe04174ef682d P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.007s
I20260618 13:59:22.360612 28800 ts_tablet_manager.cc:1929] T d6e3060118e844559803a649224d7474 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.360744 28800 log.cc:1199] T d6e3060118e844559803a649224d7474 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/d6e3060118e844559803a649224d7474
I20260618 13:59:22.361284 28225 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 37a6764a5e024903848561c9935b1a43
I20260618 13:59:22.361511 28800 ts_tablet_manager.cc:1950] T d6e3060118e844559803a649224d7474 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:22.361864 28798 raft_consensus.cc:359] T d5cb3137659a4481840fe04174ef682d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.362054 28798 raft_consensus.cc:385] T d5cb3137659a4481840fe04174ef682d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.362128 28798 raft_consensus.cc:740] T d5cb3137659a4481840fe04174ef682d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.362412 28798 consensus_queue.cc:260] T d5cb3137659a4481840fe04174ef682d P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.362560 28802 tablet_replica.cc:333] T 18fb8adfc8ee40e18a1da5f978317124 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:22.362656 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 37a6764a5e024903848561c9935b1a43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.362761 28802 raft_consensus.cc:2243] T 18fb8adfc8ee40e18a1da5f978317124 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.362911 28802 raft_consensus.cc:2272] T 18fb8adfc8ee40e18a1da5f978317124 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.362957 28798 ts_tablet_manager.cc:1434] T d5cb3137659a4481840fe04174ef682d P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260618 13:59:22.363590 28962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 13:59:22.362389 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 102a500268574e428cb2d45e52849f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.364282 28798 tablet_bootstrap.cc:492] T 8c9d89fa75174a969ded07e0596db99b P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:22.365015 28802 ts_tablet_manager.cc:1916] T 18fb8adfc8ee40e18a1da5f978317124 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.366811 28798 tablet_bootstrap.cc:654] T 8c9d89fa75174a969ded07e0596db99b P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.368013 28858 server_base.cc:1061] running on GCE node
I20260618 13:59:22.369474 28858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 13:59:22.372097 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 37a6764a5e024903848561c9935b1a43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.372177 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet 37a6764a5e024903848561c9935b1a43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.372947 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet d5cb3137659a4481840fe04174ef682d with delete_type TABLET_DATA_DELETED (replaced by tablet 68b8293a7c924a449307ab88a1cf9290) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.307427 28799 ts_tablet_manager.cc:1434] T 1f0109536c24485c9934193f2ef8aeae P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260618 13:59:22.375972 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet d6e3060118e844559803a649224d7474 (table default.Table_b35e2c7c7707422d93364bf9be3bab3f [id=625859c44f3246eaacad14ba1f027d9c]) successfully deleted
W20260618 13:59:22.376281 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 37a6764a5e024903848561c9935b1a43 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37a6764a5e024903848561c9935b1a43 already in progress: creating tablet
I20260618 13:59:22.376638 28797 tablet_bootstrap.cc:492] T 059716cec7f94ea6ad8989fecc6f832a P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:22.376775 28797 ts_tablet_manager.cc:1403] T 059716cec7f94ea6ad8989fecc6f832a P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.121s user 0.001s sys 0.002s
I20260618 13:59:22.376796 28225 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_2be2c24d66ae4f1d8bf1e78d134841c7" } modify_external_catalogs: true
I20260618 13:59:22.376948 28225 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_2be2c24d66ae4f1d8bf1e78d134841c7" } modify_external_catalogs: true
I20260618 13:59:22.377548 28797 raft_consensus.cc:359] T 059716cec7f94ea6ad8989fecc6f832a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.377705 28797 raft_consensus.cc:385] T 059716cec7f94ea6ad8989fecc6f832a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.377794 28797 raft_consensus.cc:740] T 059716cec7f94ea6ad8989fecc6f832a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.378948 28858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 13:59:22.379508 28799 tablet_bootstrap.cc:492] T b1f734a3f29b42e591d5564e99292b5e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:22.382122 28229 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_45297e5c8f754fbbb3ab4dac6de1e136" } modify_external_catalogs: true
I20260618 13:59:22.382301 28229 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_45297e5c8f754fbbb3ab4dac6de1e136" } modify_external_catalogs: true
I20260618 13:59:22.383121 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_9e1e848ca5c94952ab3928c5feafe2f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.383637 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9e1e848ca5c94952ab3928c5feafe2f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.384268 28221 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet d980cda1c60e4edca7b4d7537f4f5837
I20260618 13:59:22.384757 28221 catalog_manager.cc:6784] ReplaceTablet: tablet d980cda1c60e4edca7b4d7537f4f5837 deleted and replaced by tablet 98b42d23832545ae81a7cdff19e4689f
I20260618 13:59:22.387602 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet d980cda1c60e4edca7b4d7537f4f5837 with delete_type TABLET_DATA_DELETED (replaced by tablet 98b42d23832545ae81a7cdff19e4689f) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.388001 28797 consensus_queue.cc:260] T 059716cec7f94ea6ad8989fecc6f832a P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.391317 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet d980cda1c60e4edca7b4d7537f4f5837 with delete_type TABLET_DATA_DELETED (replaced by tablet 98b42d23832545ae81a7cdff19e4689f) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.392100 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet d980cda1c60e4edca7b4d7537f4f5837 with delete_type TABLET_DATA_DELETED (replaced by tablet 98b42d23832545ae81a7cdff19e4689f) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.392913 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 059716cec7f94ea6ad8989fecc6f832a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
W20260618 13:59:22.393199 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d980cda1c60e4edca7b4d7537f4f5837 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d980cda1c60e4edca7b4d7537f4f5837 already in progress: creating tablet
W20260618 13:59:22.393751 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 059716cec7f94ea6ad8989fecc6f832a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 059716cec7f94ea6ad8989fecc6f832a already in progress: creating tablet
I20260618 13:59:22.394331 28802 ts_tablet_manager.cc:1929] T 18fb8adfc8ee40e18a1da5f978317124 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.395114 28858 hybrid_clock.cc:648] HybridClock initialized: now 1781791162393846 us; error 1266 us; skew 500 ppm
I20260618 13:59:22.395723 28858 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 13:59:22.396512 28797 ts_tablet_manager.cc:1434] T 059716cec7f94ea6ad8989fecc6f832a P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.019s user 0.001s sys 0.001s
I20260618 13:59:22.397096 28801 ts_tablet_manager.cc:1929] T ee783d2c7e274158b3de2cc186f50903 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.397248 28801 log.cc:1199] T ee783d2c7e274158b3de2cc186f50903 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/ee783d2c7e274158b3de2cc186f50903
I20260618 13:59:22.397521 28802 log.cc:1199] T 18fb8adfc8ee40e18a1da5f978317124 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/18fb8adfc8ee40e18a1da5f978317124
I20260618 13:59:22.399119 28801 ts_tablet_manager.cc:1950] T ee783d2c7e274158b3de2cc186f50903 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:22.387811 28799 tablet_bootstrap.cc:654] T b1f734a3f29b42e591d5564e99292b5e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.400035 28225 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 397c7abe26f4451298558d6a0875b677
I20260618 13:59:22.401104 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet 397c7abe26f4451298558d6a0875b677 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.402189 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 397c7abe26f4451298558d6a0875b677 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.402513 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 397c7abe26f4451298558d6a0875b677 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.402803 28800 tablet_replica.cc:333] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:22.403015 28800 raft_consensus.cc:2243] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:22.403404 28797 tablet_bootstrap.cc:492] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:22.403587 28800 raft_consensus.cc:2272] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.404536 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 059716cec7f94ea6ad8989fecc6f832a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.404641 28802 ts_tablet_manager.cc:1950] T 18fb8adfc8ee40e18a1da5f978317124 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:22.405787 28802 tablet_replica.cc:333] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:22.405925 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 18fb8adfc8ee40e18a1da5f978317124 (table default.Table_bf7cbaec816a4c04a225ff31f57123ed [id=f34f77924d464341afa518830ba7bbbe]) successfully deleted
I20260618 13:59:22.405577 28800 ts_tablet_manager.cc:1916] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.408749 28225 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_9cd59394f11d4937887809241a69bba3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:22.409024 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet ee783d2c7e274158b3de2cc186f50903 (table default.Table_dbc49898f77e441cb501871dfae61db4 [id=d8ef984c664f4a6598c70a48e9aff826]) successfully deleted
W20260618 13:59:22.409698 28225 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9cd59394f11d4937887809241a69bba3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.410596 28801 tablet_replica.cc:333] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:22.410820 28801 raft_consensus.cc:2243] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.410983 28801 raft_consensus.cc:2272] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.413806 28798 tablet_bootstrap.cc:492] T 8c9d89fa75174a969ded07e0596db99b P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:22.413966 28798 ts_tablet_manager.cc:1403] T 8c9d89fa75174a969ded07e0596db99b P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.051s user 0.024s sys 0.021s
I20260618 13:59:22.414006 28797 tablet_bootstrap.cc:654] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.414688 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_659a3d495ed44be4bfb2193816362e25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:22.414942 28802 raft_consensus.cc:2243] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.414934 28798 raft_consensus.cc:359] T 8c9d89fa75174a969ded07e0596db99b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:22.415115 28802 raft_consensus.cc:2272] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.415117 28798 raft_consensus.cc:385] T 8c9d89fa75174a969ded07e0596db99b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260618 13:59:22.415172 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_659a3d495ed44be4bfb2193816362e25 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.415197 28798 raft_consensus.cc:740] T 8c9d89fa75174a969ded07e0596db99b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.415493 28798 consensus_queue.cc:260] T 8c9d89fa75174a969ded07e0596db99b P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:22.416738 28801 ts_tablet_manager.cc:1916] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.421144 28799 tablet_bootstrap.cc:492] T b1f734a3f29b42e591d5564e99292b5e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:22.421300 28799 ts_tablet_manager.cc:1403] T b1f734a3f29b42e591d5564e99292b5e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.042s user 0.005s sys 0.001s
I20260618 13:59:22.422128 28799 raft_consensus.cc:359] T b1f734a3f29b42e591d5564e99292b5e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.422317 28799 raft_consensus.cc:385] T b1f734a3f29b42e591d5564e99292b5e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.422449 28799 raft_consensus.cc:740] T b1f734a3f29b42e591d5564e99292b5e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.424902 28799 consensus_queue.cc:260] T b1f734a3f29b42e591d5564e99292b5e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.425958 28799 ts_tablet_manager.cc:1434] T b1f734a3f29b42e591d5564e99292b5e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260618 13:59:22.426012 28798 ts_tablet_manager.cc:1434] T 8c9d89fa75174a969ded07e0596db99b P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260618 13:59:22.425518 28802 ts_tablet_manager.cc:1916] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.427930 28798 tablet_bootstrap.cc:492] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:22.429710 28798 tablet_bootstrap.cc:654] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.426257 28799 tablet_bootstrap.cc:492] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:22.433948 28799 tablet_bootstrap.cc:654] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.435237 28858 webserver.cc:533] Webserver started at http://127.27.100.61:35663/ using document root <none> and password file <none>
I20260618 13:59:22.436564 28858 fs_manager.cc:362] Metadata directory not provided
I20260618 13:59:22.436721 28858 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 13:59:22.437616 28798 tablet_bootstrap.cc:492] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:22.437762 28798 ts_tablet_manager.cc:1403] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.008s
I20260618 13:59:22.438603 28798 raft_consensus.cc:359] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:22.438817 28798 raft_consensus.cc:385] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.438905 28798 raft_consensus.cc:740] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.439188 28798 consensus_queue.cc:260] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:22.441602 28801 ts_tablet_manager.cc:1929] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.442020 28798 ts_tablet_manager.cc:1434] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:22.447134 28798 tablet_bootstrap.cc:492] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:22.450713 28798 tablet_bootstrap.cc:654] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.453156 28801 log.cc:1199] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/144aa8984a44410fb3f90dc99e9b2274
I20260618 13:59:22.453666 28798 tablet_bootstrap.cc:492] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:22.453836 28798 ts_tablet_manager.cc:1403] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260618 13:59:22.454995 28798 raft_consensus.cc:359] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:22.455255 28798 raft_consensus.cc:385] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.455367 28798 raft_consensus.cc:740] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.455376 28801 ts_tablet_manager.cc:1950] T 144aa8984a44410fb3f90dc99e9b2274 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:22.455651 28798 consensus_queue.cc:260] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:22.456352 28798 ts_tablet_manager.cc:1434] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:22.456648 28798 tablet_bootstrap.cc:492] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:22.458303 28798 tablet_bootstrap.cc:654] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.461018 28802 ts_tablet_manager.cc:1929] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.461588 28801 tablet_replica.cc:333] T a2ca1e22c38e49d793d413e9eaaac06f P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:22.461855 28801 raft_consensus.cc:2243] T a2ca1e22c38e49d793d413e9eaaac06f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.462339 28801 raft_consensus.cc:2272] T a2ca1e22c38e49d793d413e9eaaac06f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.463704 28801 ts_tablet_manager.cc:1916] T a2ca1e22c38e49d793d413e9eaaac06f P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.463919 28802 log.cc:1199] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/ffd7d5ebaa94416a8eaf3e61bed945e1
I20260618 13:59:22.464449 28802 ts_tablet_manager.cc:1950] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:22.465701 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 144aa8984a44410fb3f90dc99e9b2274 (table default.Table_56009fb08789451f82400e86054dd066 [id=c2109218729e47e0b1057c9802e312a4]) successfully deleted
W20260618 13:59:22.465965 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 8be4374ae7cf4907940ae90299bf9ba0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8be4374ae7cf4907940ae90299bf9ba0 already in progress: creating tablet
I20260618 13:59:22.465991 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c9d89fa75174a969ded07e0596db99b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:22.466182 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet ba0d05ea44e1473fb4b24f61eab096e4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba0d05ea44e1473fb4b24f61eab096e4 already in progress: creating tablet
I20260618 13:59:22.466228 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet b312fbf800434ecb840dd10347efe6e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:22.466379 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d9c00003300246ada2a356e3e1cf684b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9c00003300246ada2a356e3e1cf684b already in progress: creating tablet
I20260618 13:59:22.466482 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet 15bbf81e3fbf4927b822afa0635909eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:22.466610 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet c552d1316e3340f4bcfa39618ae309b8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c552d1316e3340f4bcfa39618ae309b8 already in progress: creating tablet
I20260618 13:59:22.467156 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_3a9b1ea0c47e41babda533950a3d5ba8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.467867 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a9b1ea0c47e41babda533950a3d5ba8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.469131 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet ffd7d5ebaa94416a8eaf3e61bed945e1 (table default.Table_c18d75e5991b440caf136e9248a572da [id=5a3cb5befb614a70b4ecf8ccfedf8ecd]) successfully deleted
I20260618 13:59:22.469525 28220 master_service.cc:960] ReplaceTablet: received request to replace tablet 102a500268574e428cb2d45e52849f57 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:22.469530 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_6ad6cf4da29143ff9ad3187cfcb95f6e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.470376 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6ad6cf4da29143ff9ad3187cfcb95f6e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.471582 28220 master_service.cc:960] ReplaceTablet: received request to replace tablet 668c6c6f2034484ca5ccdd51c22dea1d from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:22.472278 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_734fac8462fe4ad19c5d3d9a92e06f0e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.472699 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_734fac8462fe4ad19c5d3d9a92e06f0e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.473958 28223 master_service.cc:960] ReplaceTablet: received request to replace tablet d980cda1c60e4edca7b4d7537f4f5837 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:22.474769 28224 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_7e5df99a84c84d66b548875d6d934656" } modify_external_catalogs: true
I20260618 13:59:22.474946 28224 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_7e5df99a84c84d66b548875d6d934656" } modify_external_catalogs: true
I20260618 13:59:22.475677 28227 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_03e5ab6f90514a49815d0f0c6c8e5f80" } new_table_name: "default.Table_03893e570e2c4d33b96437d5dbe1f241" modify_external_catalogs: true
W20260618 13:59:22.478904 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 37a6764a5e024903848561c9935b1a43 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37a6764a5e024903848561c9935b1a43 already in progress: creating tablet
I20260618 13:59:22.479519 28802 tablet_replica.cc:333] T d980cda1c60e4edca7b4d7537f4f5837 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:22.479735 28802 raft_consensus.cc:2243] T d980cda1c60e4edca7b4d7537f4f5837 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.479908 28802 raft_consensus.cc:2272] T d980cda1c60e4edca7b4d7537f4f5837 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.481719 28798 tablet_bootstrap.cc:492] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:22.481895 28798 ts_tablet_manager.cc:1403] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.010s
I20260618 13:59:22.482758 28798 raft_consensus.cc:359] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.482963 28798 raft_consensus.cc:385] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.483037 28798 raft_consensus.cc:740] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.483292 28798 consensus_queue.cc:260] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.484481 28802 ts_tablet_manager.cc:1916] T d980cda1c60e4edca7b4d7537f4f5837 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.493598 28798 ts_tablet_manager.cc:1434] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20260618 13:59:22.496253 28801 ts_tablet_manager.cc:1929] T a2ca1e22c38e49d793d413e9eaaac06f P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.496383 28801 log.cc:1199] T a2ca1e22c38e49d793d413e9eaaac06f P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/a2ca1e22c38e49d793d413e9eaaac06f
I20260618 13:59:22.496747 28801 ts_tablet_manager.cc:1950] T a2ca1e22c38e49d793d413e9eaaac06f P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:22.498212 28798 tablet_bootstrap.cc:492] T 85a3b27d914a412498919a3f65e20522 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:22.499359 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_37a0a01f533341e78854144439ab6dc3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.499871 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_37a0a01f533341e78854144439ab6dc3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.504246 28798 tablet_bootstrap.cc:654] T 85a3b27d914a412498919a3f65e20522 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.508708 28802 ts_tablet_manager.cc:1929] T d980cda1c60e4edca7b4d7537f4f5837 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.508880 28802 log.cc:1199] T d980cda1c60e4edca7b4d7537f4f5837 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/d980cda1c60e4edca7b4d7537f4f5837
I20260618 13:59:22.509316 28802 ts_tablet_manager.cc:1950] T d980cda1c60e4edca7b4d7537f4f5837 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:22.509996 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_bf5ce6c6e0b64002994fb45e76ab30eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:22.511264 28798 tablet_bootstrap.cc:492] T 85a3b27d914a412498919a3f65e20522 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:22.511448 28798 ts_tablet_manager.cc:1403] T 85a3b27d914a412498919a3f65e20522 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.003s
I20260618 13:59:22.511438 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet b1f734a3f29b42e591d5564e99292b5e with delete_type TABLET_DATA_DELETED (replaced by tablet e025b64380314546af8a1ab90c817825) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.512482 28798 raft_consensus.cc:359] T 85a3b27d914a412498919a3f65e20522 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:22.512624 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bf5ce6c6e0b64002994fb45e76ab30eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.512718 28798 raft_consensus.cc:385] T 85a3b27d914a412498919a3f65e20522 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.512881 28798 raft_consensus.cc:740] T 85a3b27d914a412498919a3f65e20522 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.513541 28801 tablet_replica.cc:333] T 570d36a27b7d417989ad72885d38ff63 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:22.513758 28801 raft_consensus.cc:2243] T 570d36a27b7d417989ad72885d38ff63 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.513927 28801 raft_consensus.cc:2272] T 570d36a27b7d417989ad72885d38ff63 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.515209 28801 ts_tablet_manager.cc:1916] T 570d36a27b7d417989ad72885d38ff63 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.515661 28798 consensus_queue.cc:260] T 85a3b27d914a412498919a3f65e20522 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:22.517827 28798 ts_tablet_manager.cc:1434] T 85a3b27d914a412498919a3f65e20522 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.006s user 0.001s sys 0.002s
I20260618 13:59:22.519070 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet d980cda1c60e4edca7b4d7537f4f5837 (table default.Table_3bd5981b65e545459eb539217e8e3599 [id=7c39b3c432e944bca05f7b2070a79ff9]) successfully deleted
I20260618 13:59:22.519336 28798 tablet_bootstrap.cc:492] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:22.521587 28802 tablet_replica.cc:333] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:22.521806 28802 raft_consensus.cc:2243] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.522095 28802 raft_consensus.cc:2272] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.525451 28391 tablet_service.cc:1511] Processing CreateTablet for tablet 1829d1b69cdd4e748c5901c8e02cac2e (DEFAULT_TABLE table=default.Table_e601511ed59d407b86040479045e84ec [id=8721763b01de4d54980bae1f82434853]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.525883 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1829d1b69cdd4e748c5901c8e02cac2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.525914 28802 ts_tablet_manager.cc:1916] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.531540 28798 tablet_bootstrap.cc:654] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.534106 28381 tablet_service.cc:1511] Processing CreateTablet for tablet b16f7d3c7837498e996bba1818110642 (DEFAULT_TABLE table=default.Table_061bec38727b4ee69096fe3b50296f94 [id=5fe80671a240424da5ef1d7958caba2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.534576 28381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b16f7d3c7837498e996bba1818110642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.543438 28946 raft_consensus.cc:493] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:22.543645 28946 raft_consensus.cc:515] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.544761 28946 leader_election.cc:290] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:22.545423 28802 ts_tablet_manager.cc:1929] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:22.545544 28802 log.cc:1199] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/397c7abe26f4451298558d6a0875b677
I20260618 13:59:22.545989 28802 ts_tablet_manager.cc:1950] T 397c7abe26f4451298558d6a0875b677 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:22.546192 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bf263d20934ed6a6efb4b215252c00" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:22.546831 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bf263d20934ed6a6efb4b215252c00" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:22.547144 28683 raft_consensus.cc:2468] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
W20260618 13:59:22.548560 28346 leader_election.cc:343] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:22.548841 28346 leader_election.cc:304] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:22.549436 28946 raft_consensus.cc:2804] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:22.549566 28946 raft_consensus.cc:493] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:22.549690 28946 raft_consensus.cc:3060] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:22.551227 28215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.62:43001, user_credentials={real_user=slave}} blocked reactor thread for 57396.6us
I20260618 13:59:22.551962 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet eeffc96b9b97441d9301e4919d0047e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.552652 28651 tablet_service.cc:1511] Processing CreateTablet for tablet 1829d1b69cdd4e748c5901c8e02cac2e (DEFAULT_TABLE table=default.Table_e601511ed59d407b86040479045e84ec [id=8721763b01de4d54980bae1f82434853]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.553053 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1829d1b69cdd4e748c5901c8e02cac2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.553020 28655 tablet_service.cc:1511] Processing CreateTablet for tablet b16f7d3c7837498e996bba1818110642 (DEFAULT_TABLE table=default.Table_061bec38727b4ee69096fe3b50296f94 [id=5fe80671a240424da5ef1d7958caba2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.553031 28519 tablet_service.cc:1511] Processing CreateTablet for tablet 1829d1b69cdd4e748c5901c8e02cac2e (DEFAULT_TABLE table=default.Table_e601511ed59d407b86040479045e84ec [id=8721763b01de4d54980bae1f82434853]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.553323 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b16f7d3c7837498e996bba1818110642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.553527 28519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1829d1b69cdd4e748c5901c8e02cac2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.558911 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet a2ca1e22c38e49d793d413e9eaaac06f (table default.Table_5224a46184b74b75a0379955524146f3 [id=f071d496d807405cb0f3b022757b85e0]) successfully deleted
W20260618 13:59:22.559167 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 5332b6bb76ea419086341738fd135b24 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5332b6bb76ea419086341738fd135b24 already in progress: creating tablet
I20260618 13:59:22.559221 28520 tablet_service.cc:1511] Processing CreateTablet for tablet b16f7d3c7837498e996bba1818110642 (DEFAULT_TABLE table=default.Table_061bec38727b4ee69096fe3b50296f94 [id=5fe80671a240424da5ef1d7958caba2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.559612 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b16f7d3c7837498e996bba1818110642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.560290 28946 raft_consensus.cc:515] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.561638 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bf263d20934ed6a6efb4b215252c00" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:22.562288 28346 leader_election.cc:343] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:22.566584 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bf263d20934ed6a6efb4b215252c00" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:22.566852 28683 raft_consensus.cc:3060] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:22.570338 28683 raft_consensus.cc:2468] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:22.571938 28346 leader_election.cc:304] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:22.572265 28934 raft_consensus.cc:2804] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:22.572368 28946 leader_election.cc:290] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:22.572451 28934 raft_consensus.cc:697] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:22.572716 28934 consensus_queue.cc:237] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.573089 28391 tablet_service.cc:1511] Processing CreateTablet for tablet 751cd9bd961b4f56a588ed02a05f0212 (DEFAULT_TABLE table=default.Table_74620afb05b24bf9912f3984efd632aa [id=ad88f16ed9e143cda1e892de63a18849]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.573693 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 751cd9bd961b4f56a588ed02a05f0212. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.577812 28381 tablet_service.cc:1511] Processing CreateTablet for tablet d7a97176f72349cda221baaea61d18ad (DEFAULT_TABLE table=default.Table_b0e3e2341f764392af42b949d03dd1f6 [id=27b6a19a28f24f2b8a758bbff06312d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.578902 28381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7a97176f72349cda221baaea61d18ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.590065 28655 tablet_service.cc:1511] Processing CreateTablet for tablet 751cd9bd961b4f56a588ed02a05f0212 (DEFAULT_TABLE table=default.Table_74620afb05b24bf9912f3984efd632aa [id=ad88f16ed9e143cda1e892de63a18849]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.590493 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 751cd9bd961b4f56a588ed02a05f0212. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.597308 28519 tablet_service.cc:1511] Processing CreateTablet for tablet d7a97176f72349cda221baaea61d18ad (DEFAULT_TABLE table=default.Table_b0e3e2341f764392af42b949d03dd1f6 [id=27b6a19a28f24f2b8a758bbff06312d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.597733 28519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7a97176f72349cda221baaea61d18ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.599519 28522 tablet_service.cc:1511] Processing CreateTablet for tablet bc0adedecee4405e86b0a37b9e62e352 (DEFAULT_TABLE table=default.Table_cc20b3f971bb46969baf1fc71e217f2c [id=0a955d9a7fec4cd7985ac56e210fe8a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.599923 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc0adedecee4405e86b0a37b9e62e352. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:22.602560 28227 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b0c9d4131ea24809ac081e7a73a21e1c consensus state has no leader
I20260618 13:59:22.609607 28225 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_d202b6dfbdee4280bdc0bc6e2c7094be" } new_table_name: "default.Table_4de8ba099efe4bb3bc0a0d90e1861107" modify_external_catalogs: true
I20260618 13:59:22.611047 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_d16c53f18154437e8d06e88da66faf13"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.611588 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d16c53f18154437e8d06e88da66faf13 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.612859 28227 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_77ced729210e46da993ea9e57db1d716" } modify_external_catalogs: true
I20260618 13:59:22.613014 28227 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_77ced729210e46da993ea9e57db1d716" } modify_external_catalogs: true
I20260618 13:59:22.613291 28520 tablet_service.cc:1511] Processing CreateTablet for tablet 751cd9bd961b4f56a588ed02a05f0212 (DEFAULT_TABLE table=default.Table_74620afb05b24bf9912f3984efd632aa [id=ad88f16ed9e143cda1e892de63a18849]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.613682 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 751cd9bd961b4f56a588ed02a05f0212. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.614300 28221 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_f5abc4ffe2a848fb8805a523adf0f735" } new_table_name: "default.Table_a5e98286eedd40debca5bb4f5e2ece2a" modify_external_catalogs: true
I20260618 13:59:22.614646 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_35260f90f6814fa3915b4280cff96d1b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.615141 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_35260f90f6814fa3915b4280cff96d1b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.617297 28224 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 233040a16b0d4b61b977ca57c1afec2d
I20260618 13:59:22.619253 28224 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_d52dc1e9a80f4c7db865aaf0c51bdd60" } new_table_name: "default.Table_cb42691367014b83a1ecbc738cbe0ab9" modify_external_catalogs: true
I20260618 13:59:22.620399 28229 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 4d6b01b897134d5bbd6a7d93b82ddb5e
I20260618 13:59:22.621454 28229 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_9b2c93a93eb1461994ae0ff79108098b" } new_table_name: "default.Table_74c0d9b4f3724e278144c5fc30d8c2d7" modify_external_catalogs: true
I20260618 13:59:22.629313 28810 raft_consensus.cc:493] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:22.629513 28810 raft_consensus.cc:515] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:22.630445 28810 leader_election.cc:290] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:22.636129 28223 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_e4c2db87fb024027b4a6e457a8524f10" } modify_external_catalogs: true
I20260618 13:59:22.636444 28223 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_e4c2db87fb024027b4a6e457a8524f10" } modify_external_catalogs: true
I20260618 13:59:22.638685 28226 master_service.cc:960] ReplaceTablet: received request to replace tablet 4a01dfb4d59549399c94640f16045555 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:22.642774 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_cd647e83c17a49219bc44d240cb9fb66"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.643291 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cd647e83c17a49219bc44d240cb9fb66 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:22.643491 28483 leader_election.cc:343] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: b312fbf800434ecb840dd10347efe6e8
I20260618 13:59:22.644744 28226 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_dbc49898f77e441cb501871dfae61db4" } modify_external_catalogs: true
I20260618 13:59:22.644891 28226 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_dbc49898f77e441cb501871dfae61db4" } modify_external_catalogs: true
I20260618 13:59:22.648204 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet 233040a16b0d4b61b977ca57c1afec2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.648520 28382 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.648658 28802 tablet_replica.cc:333] T b1f734a3f29b42e591d5564e99292b5e P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:22.668134 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet 85a3b27d914a412498919a3f65e20522 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:19 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.668555 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet 233040a16b0d4b61b977ca57c1afec2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.669119 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.682578 28802 raft_consensus.cc:2243] T b1f734a3f29b42e591d5564e99292b5e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.683208 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 397c7abe26f4451298558d6a0875b677 (table default.Table_2be2c24d66ae4f1d8bf1e78d134841c7 [id=ce780967c12a412a8d7f0bef2f9c00e3]) successfully deleted
I20260618 13:59:22.686990 28382 tablet_service.cc:1511] Processing CreateTablet for tablet bc0adedecee4405e86b0a37b9e62e352 (DEFAULT_TABLE table=default.Table_cc20b3f971bb46969baf1fc71e217f2c [id=0a955d9a7fec4cd7985ac56e210fe8a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.687405 28382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc0adedecee4405e86b0a37b9e62e352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.688437 28801 ts_tablet_manager.cc:1929] T 570d36a27b7d417989ad72885d38ff63 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.688596 28801 log.cc:1199] T 570d36a27b7d417989ad72885d38ff63 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/570d36a27b7d417989ad72885d38ff63
I20260618 13:59:22.689003 28801 ts_tablet_manager.cc:1950] T 570d36a27b7d417989ad72885d38ff63 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:22.686589 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 9b75ece59d924e3db49e002bf0faa89c (DEFAULT_TABLE table=default.Table_de7bace9efe6428da6ea8e662f45c90e [id=387d3ff1762749efad926181fe4c746a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.689069 28800 ts_tablet_manager.cc:1929] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260618 13:59:22.690881 28484 leader_election.cc:343] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: b312fbf800434ecb840dd10347efe6e8
I20260618 13:59:22.691093 28484 leader_election.cc:304] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:22.691349 28810 raft_consensus.cc:2749] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:22.691980 28651 tablet_service.cc:1511] Processing CreateTablet for tablet d7a97176f72349cda221baaea61d18ad (DEFAULT_TABLE table=default.Table_b0e3e2341f764392af42b949d03dd1f6 [id=27b6a19a28f24f2b8a758bbff06312d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.692417 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7a97176f72349cda221baaea61d18ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.692572 28519 tablet_service.cc:1511] Processing CreateTablet for tablet 9b75ece59d924e3db49e002bf0faa89c (DEFAULT_TABLE table=default.Table_de7bace9efe6428da6ea8e662f45c90e [id=387d3ff1762749efad926181fe4c746a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.693372 28519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b75ece59d924e3db49e002bf0faa89c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.694888 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet 233040a16b0d4b61b977ca57c1afec2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.695218 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.695694 28656 tablet_service.cc:1511] Processing CreateTablet for tablet bc0adedecee4405e86b0a37b9e62e352 (DEFAULT_TABLE table=default.Table_cc20b3f971bb46969baf1fc71e217f2c [id=0a955d9a7fec4cd7985ac56e210fe8a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.696110 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc0adedecee4405e86b0a37b9e62e352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.697186 28520 tablet_service.cc:1511] Processing CreateTablet for tablet e95c88e4b0a046a3ad69ee49af47c7c3 (DEFAULT_TABLE table=default.Table_68ab7758379b44628a68285f209716ef [id=61ce11eead544e74a31db940c3376f4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.697547 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95c88e4b0a046a3ad69ee49af47c7c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.700594 28519 tablet_service.cc:1511] Processing CreateTablet for tablet d667ea5f029f46d89fcb3be03069b97b (DEFAULT_TABLE table=default.Table_f4a03a199e51400ca4c8a5d868ec2025 [id=0ce6f37b68d8435f8f860ba43647bc3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.700953 28519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d667ea5f029f46d89fcb3be03069b97b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.702760 28385 tablet_service.cc:1511] Processing CreateTablet for tablet d667ea5f029f46d89fcb3be03069b97b (DEFAULT_TABLE table=default.Table_f4a03a199e51400ca4c8a5d868ec2025 [id=0ce6f37b68d8435f8f860ba43647bc3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.686182 28802 raft_consensus.cc:2272] T b1f734a3f29b42e591d5564e99292b5e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.706117 28654 tablet_service.cc:1511] Processing CreateTablet for tablet e95c88e4b0a046a3ad69ee49af47c7c3 (DEFAULT_TABLE table=default.Table_68ab7758379b44628a68285f209716ef [id=61ce11eead544e74a31db940c3376f4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.706342 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d667ea5f029f46d89fcb3be03069b97b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.706596 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95c88e4b0a046a3ad69ee49af47c7c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.708264 28802 ts_tablet_manager.cc:1916] T b1f734a3f29b42e591d5564e99292b5e P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.708595 28383 tablet_service.cc:1511] Processing CreateTablet for tablet c557afc3abfa4ef188bf248e89f49d87 (DEFAULT_TABLE table=default.Table_fdff0afd87584a53a884c9b76926d6d0 [id=7cd5262564144438b00bde60a6cee1e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.709106 28522 tablet_service.cc:1511] Processing CreateTablet for tablet c557afc3abfa4ef188bf248e89f49d87 (DEFAULT_TABLE table=default.Table_fdff0afd87584a53a884c9b76926d6d0 [id=7cd5262564144438b00bde60a6cee1e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.709407 28525 tablet_service.cc:1511] Processing CreateTablet for tablet f89c9954d48e407c96c294c26644e25c (DEFAULT_TABLE table=default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.709522 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c557afc3abfa4ef188bf248e89f49d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.709756 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f89c9954d48e407c96c294c26644e25c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.707978 28391 tablet_service.cc:1511] Processing CreateTablet for tablet e95c88e4b0a046a3ad69ee49af47c7c3 (DEFAULT_TABLE table=default.Table_68ab7758379b44628a68285f209716ef [id=61ce11eead544e74a31db940c3376f4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.710807 28797 tablet_bootstrap.cc:492] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:22.711334 28797 ts_tablet_manager.cc:1403] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.308s user 0.011s sys 0.018s
I20260618 13:59:22.712344 28797 raft_consensus.cc:359] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.712602 28797 raft_consensus.cc:385] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.712685 28797 raft_consensus.cc:740] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.712975 28797 consensus_queue.cc:260] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.713605 28797 ts_tablet_manager.cc:1434] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:22.714099 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 9b75ece59d924e3db49e002bf0faa89c (DEFAULT_TABLE table=default.Table_de7bace9efe6428da6ea8e662f45c90e [id=387d3ff1762749efad926181fe4c746a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.716161 28801 tablet_replica.cc:333] T 102a500268574e428cb2d45e52849f57 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:22.716337 28801 raft_consensus.cc:2243] T 102a500268574e428cb2d45e52849f57 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.716533 28801 raft_consensus.cc:2272] T 102a500268574e428cb2d45e52849f57 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.717324 28525 tablet_service.cc:1511] Processing CreateTablet for tablet e86aeb21427e4859a5266f4451469b25 (DEFAULT_TABLE table=default.Table_7e0256d5218f469c9fef22ba76bb4e64 [id=65db0daf3f084f09bfd0bba3011741d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.717708 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e86aeb21427e4859a5266f4451469b25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.718546 28650 tablet_service.cc:1511] Processing CreateTablet for tablet d667ea5f029f46d89fcb3be03069b97b (DEFAULT_TABLE table=default.Table_f4a03a199e51400ca4c8a5d868ec2025 [id=0ce6f37b68d8435f8f860ba43647bc3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.719681 28659 tablet_service.cc:1511] Processing CreateTablet for tablet f89c9954d48e407c96c294c26644e25c (DEFAULT_TABLE table=default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.720284 28661 tablet_service.cc:1511] Processing CreateTablet for tablet e86aeb21427e4859a5266f4451469b25 (DEFAULT_TABLE table=default.Table_7e0256d5218f469c9fef22ba76bb4e64 [id=65db0daf3f084f09bfd0bba3011741d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.721566 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b75ece59d924e3db49e002bf0faa89c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.722604 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 570d36a27b7d417989ad72885d38ff63 (table default.Table_7d3c22c138704d71a7f2a0e756cfa3a2 [id=e38a333ca62d42518ab53a9ac17df2b7]) successfully deleted
W20260618 13:59:22.722857 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 36b8a46a41e543c58a946b6764618c7d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36b8a46a41e543c58a946b6764618c7d already in progress: creating tablet
W20260618 13:59:22.723155 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 7e5517e820ae47de810ff817f5382839 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e5517e820ae47de810ff817f5382839 already in progress: creating tablet
W20260618 13:59:22.723323 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet edc72a06ee8e4420b07a17d2feb3fcc2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet edc72a06ee8e4420b07a17d2feb3fcc2 already in progress: creating tablet
W20260618 13:59:22.723496 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet ac58e939730c4af681c2f08bcfbb6602 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ac58e939730c4af681c2f08bcfbb6602 already in progress: creating tablet
I20260618 13:59:22.725277 28390 tablet_service.cc:1511] Processing CreateTablet for tablet e86aeb21427e4859a5266f4451469b25 (DEFAULT_TABLE table=default.Table_7e0256d5218f469c9fef22ba76bb4e64 [id=65db0daf3f084f09bfd0bba3011741d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.726676 28801 ts_tablet_manager.cc:1916] T 102a500268574e428cb2d45e52849f57 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.727351 28797 tablet_bootstrap.cc:492] T 18fb8adfc8ee40e18a1da5f978317124 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:22.727445 28390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e86aeb21427e4859a5266f4451469b25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.728374 28858 fs_manager.cc:714] Time spent opening directory manager: real 0.286s user 0.077s sys 0.172s
I20260618 13:59:22.732748 28222 catalog_manager.cc:5696] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:22.733117 28222 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 29bf263d20934ed6a6efb4b215252c00 (table default.Table_5f8fbc1f56c1484cb99b55730190bbcf [id=7923680bde144e3dbb78c922d09540b9]). Expected version 4 got 0
I20260618 13:59:22.733781 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b75ece59d924e3db49e002bf0faa89c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.733969 28227 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet dca54111fea047d9ae0fabe376d89382
I20260618 13:59:22.734499 28388 tablet_service.cc:1511] Processing CreateTablet for tablet f89c9954d48e407c96c294c26644e25c (DEFAULT_TABLE table=default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.735646 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95c88e4b0a046a3ad69ee49af47c7c3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:22.736156 28221 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 41107eb1d9164a9c98060003b4027412 consensus state has no leader
I20260618 13:59:22.736549 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f89c9954d48e407c96c294c26644e25c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:22.736536 28224 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1f0109536c24485c9934193f2ef8aeae consensus state has no leader
W20260618 13:59:22.736894 28225 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a1a06078974e4bfbbc4b949802a39bf1 consensus state has no leader
I20260618 13:59:22.737771 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c557afc3abfa4ef188bf248e89f49d87. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:22.737955 28229 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8b59255b14df4158afe37cb8890e7415 consensus state has no leader
I20260618 13:59:22.738518 28661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e86aeb21427e4859a5266f4451469b25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.739225 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet dca54111fea047d9ae0fabe376d89382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.744288 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f89c9954d48e407c96c294c26644e25c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.746315 28229 master_service.cc:960] ReplaceTablet: received request to replace tablet e025b64380314546af8a1ab90c817825 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:22.748663 28225 master_service.cc:960] ReplaceTablet: received request to replace tablet 1d23b5d2d67945a6ab9591969acb0d27 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:22.750537 28658 tablet_service.cc:1511] Processing CreateTablet for tablet c557afc3abfa4ef188bf248e89f49d87 (DEFAULT_TABLE table=default.Table_fdff0afd87584a53a884c9b76926d6d0 [id=7cd5262564144438b00bde60a6cee1e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.752140 28650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d667ea5f029f46d89fcb3be03069b97b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.755082 28658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c557afc3abfa4ef188bf248e89f49d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.755630 28982 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 13:59:22.756865 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet dca54111fea047d9ae0fabe376d89382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.757138 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d23b5d2d67945a6ab9591969acb0d27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.757164 28800 log.cc:1199] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/397c7abe26f4451298558d6a0875b677
I20260618 13:59:22.757920 28858 fs_manager.cc:730] Time spent opening block manager: real 0.027s user 0.012s sys 0.011s
I20260618 13:59:22.758064 28653 tablet_service.cc:1511] Processing CreateTablet for tablet dc36fbea853a4d4e9f61909200239aed (DEFAULT_TABLE table=default.Table_454932cc31ee4f6d8515153fce3b4666 [id=49576dabdecf41aca1b15ff3fc0c113c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.758102 28858 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/data,/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/wal
uuid: "a950ab7cd66b4cb0b29592aa692c2dd0"
format_stamp: "Formatted at 2026-06-18 13:59:16 on dist-test-slave-bmq2"
I20260618 13:59:22.759351 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet dca54111fea047d9ae0fabe376d89382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.759677 28525 tablet_service.cc:1511] Processing CreateTablet for tablet dc36fbea853a4d4e9f61909200239aed (DEFAULT_TABLE table=default.Table_454932cc31ee4f6d8515153fce3b4666 [id=49576dabdecf41aca1b15ff3fc0c113c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.760104 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc36fbea853a4d4e9f61909200239aed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.759430 28858 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 13:59:22.760691 28223 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 94365c44b5e24acab5df60c67203794a
I20260618 13:59:22.761427 28226 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 311ba4dd83984f379c8f4f8274bc26c1
I20260618 13:59:22.762687 28519 tablet_service.cc:1511] Processing CreateTablet for tablet aa8cf927bed9437086efd8e0f21ba26a (DEFAULT_TABLE table=default.Table_cf80fd5bbdde4cfb855c8fbe860a15f3 [id=9d63ad7327a44124af817b465a9ae0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.763005 28519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa8cf927bed9437086efd8e0f21ba26a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.766988 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet 94365c44b5e24acab5df60c67203794a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.767318 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet 311ba4dd83984f379c8f4f8274bc26c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.769896 28798 tablet_bootstrap.cc:492] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:22.770323 28798 ts_tablet_manager.cc:1403] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.252s user 0.002s sys 0.006s
I20260618 13:59:22.770387 28229 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet e025b64380314546af8a1ab90c817825
I20260618 13:59:22.770915 28229 catalog_manager.cc:6784] ReplaceTablet: tablet e025b64380314546af8a1ab90c817825 deleted and replaced by tablet ece7909d056c4ba8a5dca6423f0c1781
I20260618 13:59:22.772398 28934 raft_consensus.cc:493] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:22.772595 28934 raft_consensus.cc:515] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.772660 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet e025b64380314546af8a1ab90c817825 with delete_type TABLET_DATA_DELETED (replaced by tablet ece7909d056c4ba8a5dca6423f0c1781) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.773756 28934 leader_election.cc:290] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:22.773942 28387 tablet_service.cc:1558] Processing DeleteTablet for tablet 94365c44b5e24acab5df60c67203794a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.774236 28386 tablet_service.cc:1511] Processing CreateTablet for tablet aa8cf927bed9437086efd8e0f21ba26a (DEFAULT_TABLE table=default.Table_cf80fd5bbdde4cfb855c8fbe860a15f3 [id=9d63ad7327a44124af817b465a9ae0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.774878 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa8cf927bed9437086efd8e0f21ba26a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.776007 28381 tablet_service.cc:1511] Processing CreateTablet for tablet dc36fbea853a4d4e9f61909200239aed (DEFAULT_TABLE table=default.Table_454932cc31ee4f6d8515153fce3b4666 [id=49576dabdecf41aca1b15ff3fc0c113c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.776651 28381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc36fbea853a4d4e9f61909200239aed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.777302 28387 tablet_service.cc:1558] Processing DeleteTablet for tablet 311ba4dd83984f379c8f4f8274bc26c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.778378 28387 tablet_service.cc:1558] Processing DeleteTablet for tablet e025b64380314546af8a1ab90c817825 with delete_type TABLET_DATA_DELETED (replaced by tablet ece7909d056c4ba8a5dca6423f0c1781) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.778980 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311ba4dd83984f379c8f4f8274bc26c1" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:22.779549 28346 leader_election.cc:343] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:22.782696 28798 raft_consensus.cc:359] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.782963 28798 raft_consensus.cc:385] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.783056 28798 raft_consensus.cc:740] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.783347 28798 consensus_queue.cc:260] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.783990 28798 ts_tablet_manager.cc:1434] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20260618 13:59:22.786235 28798 tablet_bootstrap.cc:492] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:22.793452 28222 master_service.cc:960] ReplaceTablet: received request to replace tablet 2188ebf1727240cd886ba7662f8fcf43 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:22.794413 28798 tablet_bootstrap.cc:654] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.755450 28652 tablet_service.cc:1511] Processing CreateTablet for tablet aa8cf927bed9437086efd8e0f21ba26a (DEFAULT_TABLE table=default.Table_cf80fd5bbdde4cfb855c8fbe860a15f3 [id=9d63ad7327a44124af817b465a9ae0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.804615 28802 ts_tablet_manager.cc:1929] T b1f734a3f29b42e591d5564e99292b5e P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.808777 28802 log.cc:1199] T b1f734a3f29b42e591d5564e99292b5e P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/b1f734a3f29b42e591d5564e99292b5e
I20260618 13:59:22.809286 28802 ts_tablet_manager.cc:1950] T b1f734a3f29b42e591d5564e99292b5e P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:22.809361 28801 ts_tablet_manager.cc:1929] T 102a500268574e428cb2d45e52849f57 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.811664 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 311ba4dd83984f379c8f4f8274bc26c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.811894 28801 log.cc:1199] T 102a500268574e428cb2d45e52849f57 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/102a500268574e428cb2d45e52849f57
I20260618 13:59:22.812301 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc36fbea853a4d4e9f61909200239aed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.812295 28801 ts_tablet_manager.cc:1950] T 102a500268574e428cb2d45e52849f57 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:22.814062 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet b1f734a3f29b42e591d5564e99292b5e (table default.Table_c940d6e3d5d34adfaf39c7b5b60044e1 [id=803d5220ce1a4dcfaf27d93c574be2ac]) successfully deleted
W20260618 13:59:22.814617 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 233040a16b0d4b61b977ca57c1afec2d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 233040a16b0d4b61b977ca57c1afec2d already in progress: creating tablet
W20260618 13:59:22.814868 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d6b01b897134d5bbd6a7d93b82ddb5e already in progress: creating tablet
I20260618 13:59:22.815001 28802 tablet_replica.cc:333] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
W20260618 13:59:22.815085 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet dca54111fea047d9ae0fabe376d89382 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dca54111fea047d9ae0fabe376d89382 already in progress: creating tablet
I20260618 13:59:22.815191 28802 raft_consensus.cc:2243] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
W20260618 13:59:22.815313 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 94365c44b5e24acab5df60c67203794a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 94365c44b5e24acab5df60c67203794a already in progress: creating tablet
I20260618 13:59:22.815369 28802 raft_consensus.cc:2272] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.815884 28802 ts_tablet_manager.cc:1916] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.817312 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet e025b64380314546af8a1ab90c817825 with delete_type TABLET_DATA_DELETED (replaced by tablet ece7909d056c4ba8a5dca6423f0c1781) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.822749 28229 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 29bf263d20934ed6a6efb4b215252c00 (table default.Table_5f8fbc1f56c1484cb99b55730190bbcf [id=7923680bde144e3dbb78c922d09540b9]). Expected version 4 got 0
I20260618 13:59:22.828145 28222 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 2188ebf1727240cd886ba7662f8fcf43
I20260618 13:59:22.828630 28222 catalog_manager.cc:6784] ReplaceTablet: tablet 2188ebf1727240cd886ba7662f8fcf43 deleted and replaced by tablet c0a42aa779ae452e9420d730bf8d7d55
I20260618 13:59:22.829305 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 897bf6d6bbee4ad89c296325d94a3e53 (DEFAULT_TABLE table=default.Table_35260f90f6814fa3915b4280cff96d1b [id=02ffed3d852845d6845fb5176f036fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.829926 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 897bf6d6bbee4ad89c296325d94a3e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.830211 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet 2188ebf1727240cd886ba7662f8fcf43 with delete_type TABLET_DATA_DELETED (replaced by tablet c0a42aa779ae452e9420d730bf8d7d55) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.831238 28519 tablet_service.cc:1511] Processing CreateTablet for tablet 897bf6d6bbee4ad89c296325d94a3e53 (DEFAULT_TABLE table=default.Table_35260f90f6814fa3915b4280cff96d1b [id=02ffed3d852845d6845fb5176f036fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.831739 28519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 897bf6d6bbee4ad89c296325d94a3e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.833452 28391 tablet_service.cc:1511] Processing CreateTablet for tablet 283523e5b22843ba935eabe6283d3a99 (DEFAULT_TABLE table=default.Table_d16c53f18154437e8d06e88da66faf13 [id=cf2c2a972733485a9ba1d6f1e99937a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.833948 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 283523e5b22843ba935eabe6283d3a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.834106 28381 tablet_service.cc:1558] Processing DeleteTablet for tablet 2188ebf1727240cd886ba7662f8fcf43 with delete_type TABLET_DATA_DELETED (replaced by tablet c0a42aa779ae452e9420d730bf8d7d55) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.833971 28802 ts_tablet_manager.cc:1929] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:22.837265 28802 log.cc:1199] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/311ba4dd83984f379c8f4f8274bc26c1
I20260618 13:59:22.837713 28802 ts_tablet_manager.cc:1950] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:22.839085 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311ba4dd83984f379c8f4f8274bc26c1" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:22.839202 28799 tablet_bootstrap.cc:492] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:22.839341 28799 ts_tablet_manager.cc:1403] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.413s user 0.065s sys 0.133s
I20260618 13:59:22.839445 28652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa8cf927bed9437086efd8e0f21ba26a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:22.839778 28346 leader_election.cc:343] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:22.839911 28346 leader_election.cc:304] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:22.840108 28346 raft_consensus.cc:2686] T 311ba4dd83984f379c8f4f8274bc26c1 P 6fc808af0e884ed8836561665d8e0e7b: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260618 13:59:22.840583 28799 raft_consensus.cc:359] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.840780 28799 raft_consensus.cc:385] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.840847 28799 raft_consensus.cc:740] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.841078 28799 consensus_queue.cc:260] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.841231 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet 2188ebf1727240cd886ba7662f8fcf43 with delete_type TABLET_DATA_DELETED (replaced by tablet c0a42aa779ae452e9420d730bf8d7d55) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.841472 28657 tablet_service.cc:1511] Processing CreateTablet for tablet 283523e5b22843ba935eabe6283d3a99 (DEFAULT_TABLE table=default.Table_d16c53f18154437e8d06e88da66faf13 [id=cf2c2a972733485a9ba1d6f1e99937a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.841598 28799 ts_tablet_manager.cc:1434] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:22.841935 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 283523e5b22843ba935eabe6283d3a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.841946 28799 tablet_bootstrap.cc:492] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:22.843336 28655 tablet_service.cc:1511] Processing CreateTablet for tablet 897bf6d6bbee4ad89c296325d94a3e53 (DEFAULT_TABLE table=default.Table_35260f90f6814fa3915b4280cff96d1b [id=02ffed3d852845d6845fb5176f036fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.843621 28799 tablet_bootstrap.cc:654] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.843712 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 897bf6d6bbee4ad89c296325d94a3e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.845379 28229 master_service.cc:960] ReplaceTablet: received request to replace tablet 1f0109536c24485c9934193f2ef8aeae from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:22.845283 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_7cea29656b76480fb24d915e7478121a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.845922 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7cea29656b76480fb24d915e7478121a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.849043 28226 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet e1a75029992645c5b942c0ec07862bb4 (table default.Table_af2d1c7150e84f96807cda9b1dabd299 [id=bd90cbbd5592449480ef377189a9c2de]). Expected version 1 got 0
I20260618 13:59:22.852080 28798 tablet_bootstrap.cc:492] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:22.852177 28520 tablet_service.cc:1511] Processing CreateTablet for tablet 46bfe8cc2078414d84e65e22ed4ef8a2 (DEFAULT_TABLE table=default.Table_659a3d495ed44be4bfb2193816362e25 [id=2659803e822d4761a15e0e89644cc472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.852757 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46bfe8cc2078414d84e65e22ed4ef8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.857658 28229 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 1f0109536c24485c9934193f2ef8aeae
I20260618 13:59:22.858142 28229 catalog_manager.cc:6784] ReplaceTablet: tablet 1f0109536c24485c9934193f2ef8aeae deleted and replaced by tablet 6b8f273def3e4b7caa4d624cf87a106c
I20260618 13:59:22.858762 28522 tablet_service.cc:1511] Processing CreateTablet for tablet b6dd422c05324c989e29b20ae2afabad (DEFAULT_TABLE table=default.Table_734fac8462fe4ad19c5d3d9a92e06f0e [id=070ea48af78346aa8ff43f2bb369730c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.859189 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6dd422c05324c989e29b20ae2afabad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.860060 28525 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f0109536c24485c9934193f2ef8aeae with delete_type TABLET_DATA_DELETED (replaced by tablet 6b8f273def3e4b7caa4d624cf87a106c) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.860087 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f0109536c24485c9934193f2ef8aeae with delete_type TABLET_DATA_DELETED (replaced by tablet 6b8f273def3e4b7caa4d624cf87a106c) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.860553 28381 tablet_service.cc:1511] Processing CreateTablet for tablet b6dd422c05324c989e29b20ae2afabad (DEFAULT_TABLE table=default.Table_734fac8462fe4ad19c5d3d9a92e06f0e [id=070ea48af78346aa8ff43f2bb369730c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.861120 28381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6dd422c05324c989e29b20ae2afabad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.861172 28810 raft_consensus.cc:493] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:22.861321 28810 raft_consensus.cc:515] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:22.861980 28810 leader_election.cc:290] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:22.863240 28483 leader_election.cc:343] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 15bbf81e3fbf4927b822afa0635909eb
I20260618 13:59:22.864559 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 311ba4dd83984f379c8f4f8274bc26c1 (table default.Table_dbc49898f77e441cb501871dfae61db4 [id=d8ef984c664f4a6598c70a48e9aff826]) successfully deleted
I20260618 13:59:22.866127 28802 tablet_replica.cc:333] T 1f0109536c24485c9934193f2ef8aeae P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
W20260618 13:59:22.866883 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet e025b64380314546af8a1ab90c817825 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e025b64380314546af8a1ab90c817825 already in progress: creating tablet
W20260618 13:59:22.867148 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 2188ebf1727240cd886ba7662f8fcf43 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2188ebf1727240cd886ba7662f8fcf43 already in progress: creating tablet
I20260618 13:59:22.866569 28386 tablet_service.cc:1511] Processing CreateTablet for tablet b0285c75c632457383e81805ab036aaf (DEFAULT_TABLE table=default.Table_9cd59394f11d4937887809241a69bba3 [id=75fcc464e22646f7960501f112b3f887]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.867532 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0285c75c632457383e81805ab036aaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.866925 28802 raft_consensus.cc:2243] T 1f0109536c24485c9934193f2ef8aeae P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.868569 28802 raft_consensus.cc:2272] T 1f0109536c24485c9934193f2ef8aeae P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:22.869572 28484 leader_election.cc:343] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 15bbf81e3fbf4927b822afa0635909eb
I20260618 13:59:22.869737 28484 leader_election.cc:304] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:22.870023 28810 raft_consensus.cc:2749] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:22.872565 28662 tablet_service.cc:1511] Processing CreateTablet for tablet 46bfe8cc2078414d84e65e22ed4ef8a2 (DEFAULT_TABLE table=default.Table_659a3d495ed44be4bfb2193816362e25 [id=2659803e822d4761a15e0e89644cc472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.874648 28222 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_454932cc31ee4f6d8515153fce3b4666" } new_table_name: "default.Table_4e9ed3af75b34b83bca411019597566d" modify_external_catalogs: true
I20260618 13:59:22.874882 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_191169025bff42cab93cf42503fa7220"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.875422 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_191169025bff42cab93cf42503fa7220 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.875960 28221 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_68ab7758379b44628a68285f209716ef" } modify_external_catalogs: true
I20260618 13:59:22.876111 28221 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_68ab7758379b44628a68285f209716ef" } modify_external_catalogs: true
I20260618 13:59:22.876780 28228 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_e9f01ff23601409189a5923e3755f4f8" } modify_external_catalogs: true
I20260618 13:59:22.876919 28228 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_e9f01ff23601409189a5923e3755f4f8" } modify_external_catalogs: true
I20260618 13:59:22.877036 28663 tablet_service.cc:1511] Processing CreateTablet for tablet b6dd422c05324c989e29b20ae2afabad (DEFAULT_TABLE table=default.Table_734fac8462fe4ad19c5d3d9a92e06f0e [id=070ea48af78346aa8ff43f2bb369730c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.877431 28649 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f0109536c24485c9934193f2ef8aeae with delete_type TABLET_DATA_DELETED (replaced by tablet 6b8f273def3e4b7caa4d624cf87a106c) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.877490 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_d838306823ff4e22a68efe0a8cb751ff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:22.877666 28648 tablet_service.cc:1511] Processing CreateTablet for tablet b0285c75c632457383e81805ab036aaf (DEFAULT_TABLE table=default.Table_9cd59394f11d4937887809241a69bba3 [id=75fcc464e22646f7960501f112b3f887]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:22.877970 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d838306823ff4e22a68efe0a8cb751ff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.878096 28225 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_62bf5ce52fb14bd089678663dd009a91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.878594 28225 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_62bf5ce52fb14bd089678663dd009a91 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.879267 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet 94365c44b5e24acab5df60c67203794a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.879624 28229 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_5b3dc6cb553c45638fde791608253e3c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:22.879827 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_f75651a1e55f4271b39388713dcd74aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.880120 28229 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b3dc6cb553c45638fde791608253e3c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:22.880187 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f75651a1e55f4271b39388713dcd74aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.880440 28798 ts_tablet_manager.cc:1403] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.066s user 0.010s sys 0.000s
I20260618 13:59:22.882179 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_40fdb9a7e5ad4ff2bf879ec4e02a4b8d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:22.882755 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_40fdb9a7e5ad4ff2bf879ec4e02a4b8d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.883256 28797 tablet_bootstrap.cc:654] T 18fb8adfc8ee40e18a1da5f978317124 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:22.884790 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46bfe8cc2078414d84e65e22ed4ef8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.887791 28525 tablet_service.cc:1511] Processing CreateTablet for tablet b0285c75c632457383e81805ab036aaf (DEFAULT_TABLE table=default.Table_9cd59394f11d4937887809241a69bba3 [id=75fcc464e22646f7960501f112b3f887]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.888221 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0285c75c632457383e81805ab036aaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.891263 28524 tablet_service.cc:1511] Processing CreateTablet for tablet 0fae928833e44d2ba860d6184d4c0142 (DEFAULT_TABLE table=default.Table_9e1e848ca5c94952ab3928c5feafe2f1 [id=14652783e7f54ebcb5d7ba01d1855118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.891673 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fae928833e44d2ba860d6184d4c0142. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:22.892156 28222 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dc36fbea853a4d4e9f61909200239aed consensus state has no leader
I20260618 13:59:22.893801 28526 tablet_service.cc:1511] Processing CreateTablet for tablet d3dbf13544ce408ab97834aec5862390 (DEFAULT_TABLE table=default.Table_bf5ce6c6e0b64002994fb45e76ab30eb [id=a0e6904d88704db39c194c34ac9a7faf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.894165 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3dbf13544ce408ab97834aec5862390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.894791 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 827cc0987bd541759e334fb620f04ed5 (DEFAULT_TABLE table=default.Table_37a0a01f533341e78854144439ab6dc3 [id=9be95b9adc544c0e9675071330a21225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.895200 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 827cc0987bd541759e334fb620f04ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.899586 28527 tablet_service.cc:1511] Processing CreateTablet for tablet 98b42d23832545ae81a7cdff19e4689f (DEFAULT_TABLE table=default.Table_3bd5981b65e545459eb539217e8e3599 [id=7c39b3c432e944bca05f7b2070a79ff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.900008 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98b42d23832545ae81a7cdff19e4689f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.901125 28529 tablet_service.cc:1511] Processing CreateTablet for tablet 512a9e63ba324a30b8d8c37e585ebdf8 (DEFAULT_TABLE table=default.Table_3a9b1ea0c47e41babda533950a3d5ba8 [id=901874cb80e74ba589679bb7b95c8fe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.901510 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 512a9e63ba324a30b8d8c37e585ebdf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.906585 28528 tablet_service.cc:1511] Processing CreateTablet for tablet 7d4144b1b5ea4e968672616bc22bead8 (DEFAULT_TABLE table=default.Table_6ad6cf4da29143ff9ad3187cfcb95f6e [id=c2704c9f28bf4351a809b58bf4af056f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.907109 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d4144b1b5ea4e968672616bc22bead8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.917745 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 46bfe8cc2078414d84e65e22ed4ef8a2 (DEFAULT_TABLE table=default.Table_659a3d495ed44be4bfb2193816362e25 [id=2659803e822d4761a15e0e89644cc472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.921345 28384 tablet_service.cc:1511] Processing CreateTablet for tablet 512a9e63ba324a30b8d8c37e585ebdf8 (DEFAULT_TABLE table=default.Table_3a9b1ea0c47e41babda533950a3d5ba8 [id=901874cb80e74ba589679bb7b95c8fe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.924185 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6dd422c05324c989e29b20ae2afabad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.925477 28648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0285c75c632457383e81805ab036aaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.927289 28519 tablet_service.cc:1511] Processing CreateTablet for tablet 283523e5b22843ba935eabe6283d3a99 (DEFAULT_TABLE table=default.Table_d16c53f18154437e8d06e88da66faf13 [id=cf2c2a972733485a9ba1d6f1e99937a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.927676 28223 master_service.cc:960] ReplaceTablet: received request to replace tablet d7a97176f72349cda221baaea61d18ad from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:22.929308 28659 tablet_service.cc:1511] Processing CreateTablet for tablet 0fae928833e44d2ba860d6184d4c0142 (DEFAULT_TABLE table=default.Table_9e1e848ca5c94952ab3928c5feafe2f1 [id=14652783e7f54ebcb5d7ba01d1855118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.934885 28660 tablet_service.cc:1511] Processing CreateTablet for tablet d3dbf13544ce408ab97834aec5862390 (DEFAULT_TABLE table=default.Table_bf5ce6c6e0b64002994fb45e76ab30eb [id=a0e6904d88704db39c194c34ac9a7faf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.937448 28650 tablet_service.cc:1511] Processing CreateTablet for tablet 98b42d23832545ae81a7cdff19e4689f (DEFAULT_TABLE table=default.Table_3bd5981b65e545459eb539217e8e3599 [id=7c39b3c432e944bca05f7b2070a79ff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.938925 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 827cc0987bd541759e334fb620f04ed5 (DEFAULT_TABLE table=default.Table_37a0a01f533341e78854144439ab6dc3 [id=9be95b9adc544c0e9675071330a21225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.939337 28385 tablet_service.cc:1511] Processing CreateTablet for tablet 98b42d23832545ae81a7cdff19e4689f (DEFAULT_TABLE table=default.Table_3bd5981b65e545459eb539217e8e3599 [id=7c39b3c432e944bca05f7b2070a79ff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.942337 28221 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet e95c88e4b0a046a3ad69ee49af47c7c3
I20260618 13:59:22.949606 28382 tablet_service.cc:1511] Processing CreateTablet for tablet 0fae928833e44d2ba860d6184d4c0142 (DEFAULT_TABLE table=default.Table_9e1e848ca5c94952ab3928c5feafe2f1 [id=14652783e7f54ebcb5d7ba01d1855118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.950134 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 512a9e63ba324a30b8d8c37e585ebdf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.950045 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_e0fda74dbcde4ee7bc2113a6c31cc9fc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:22.950285 28222 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_74620afb05b24bf9912f3984efd632aa" } modify_external_catalogs: true
I20260618 13:59:22.950487 28222 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_74620afb05b24bf9912f3984efd632aa" } modify_external_catalogs: true
W20260618 13:59:22.950762 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e0fda74dbcde4ee7bc2113a6c31cc9fc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.951033 28382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fae928833e44d2ba860d6184d4c0142. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.952085 28227 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_8a78f34a376f49f88006dc80c596745f" } new_table_name: "default.Table_76724583c0db40378d257d673a601220" modify_external_catalogs: true
I20260618 13:59:22.953655 28387 tablet_service.cc:1511] Processing CreateTablet for tablet 7d4144b1b5ea4e968672616bc22bead8 (DEFAULT_TABLE table=default.Table_6ad6cf4da29143ff9ad3187cfcb95f6e [id=c2704c9f28bf4351a809b58bf4af056f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.954077 28387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d4144b1b5ea4e968672616bc22bead8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.957015 28228 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 37fedd27109944e2b3c870418c117758
I20260618 13:59:22.957846 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_b9f02faebbe34f9299f751f0e9f17d57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:22.958106 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46bfe8cc2078414d84e65e22ed4ef8a2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:22.958514 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b9f02faebbe34f9299f751f0e9f17d57 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:22.960983 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet e95c88e4b0a046a3ad69ee49af47c7c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.965401 28519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 283523e5b22843ba935eabe6283d3a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.965548 28658 tablet_service.cc:1558] Processing DeleteTablet for tablet e95c88e4b0a046a3ad69ee49af47c7c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.949577 28649 tablet_service.cc:1511] Processing CreateTablet for tablet 512a9e63ba324a30b8d8c37e585ebdf8 (DEFAULT_TABLE table=default.Table_3a9b1ea0c47e41babda533950a3d5ba8 [id=901874cb80e74ba589679bb7b95c8fe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.966480 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 827cc0987bd541759e334fb620f04ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.951498 28525 tablet_service.cc:1558] Processing DeleteTablet for tablet e95c88e4b0a046a3ad69ee49af47c7c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.960515 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet 37fedd27109944e2b3c870418c117758 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:22.968253 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fae928833e44d2ba860d6184d4c0142. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.968535 28810 raft_consensus.cc:493] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:22.968705 28810 raft_consensus.cc:515] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.968868 28800 ts_tablet_manager.cc:1950] T 397c7abe26f4451298558d6a0875b677 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:22.969463 28810 leader_election.cc:290] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:22.970404 28484 leader_election.cc:343] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: eeffc96b9b97441d9301e4919d0047e1
W20260618 13:59:22.970572 28483 leader_election.cc:343] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: eeffc96b9b97441d9301e4919d0047e1
I20260618 13:59:22.970697 28484 leader_election.cc:304] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:22.971108 28810 raft_consensus.cc:2749] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:22.974192 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98b42d23832545ae81a7cdff19e4689f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.975085 28388 tablet_service.cc:1511] Processing CreateTablet for tablet d3dbf13544ce408ab97834aec5862390 (DEFAULT_TABLE table=default.Table_bf5ce6c6e0b64002994fb45e76ab30eb [id=a0e6904d88704db39c194c34ac9a7faf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.978395 28382 tablet_service.cc:1558] Processing DeleteTablet for tablet 37fedd27109944e2b3c870418c117758 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:22.979274 28802 ts_tablet_manager.cc:1916] T 1f0109536c24485c9934193f2ef8aeae P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.982272 28656 tablet_service.cc:1511] Processing CreateTablet for tablet 827cc0987bd541759e334fb620f04ed5 (DEFAULT_TABLE table=default.Table_37a0a01f533341e78854144439ab6dc3 [id=9be95b9adc544c0e9675071330a21225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.982550 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 397c7abe26f4451298558d6a0875b677 (table default.Table_2be2c24d66ae4f1d8bf1e78d134841c7 [id=ce780967c12a412a8d7f0bef2f9c00e3]) successfully deleted
I20260618 13:59:22.985605 28661 tablet_service.cc:1558] Processing DeleteTablet for tablet 37fedd27109944e2b3c870418c117758 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:22.986070 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3dbf13544ce408ab97834aec5862390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:22.988159 28648 tablet_service.cc:1511] Processing CreateTablet for tablet 7d4144b1b5ea4e968672616bc22bead8 (DEFAULT_TABLE table=default.Table_6ad6cf4da29143ff9ad3187cfcb95f6e [id=c2704c9f28bf4351a809b58bf4af056f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:22.991138 28800 tablet_replica.cc:333] T 059716cec7f94ea6ad8989fecc6f832a P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:22.991484 28798 raft_consensus.cc:359] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.991766 28798 raft_consensus.cc:385] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:22.991691 28800 raft_consensus.cc:2243] T 059716cec7f94ea6ad8989fecc6f832a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:22.991878 28798 raft_consensus.cc:740] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:22.992177 28800 raft_consensus.cc:2272] T 059716cec7f94ea6ad8989fecc6f832a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:22.992676 28798 consensus_queue.cc:260] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:22.993666 28800 ts_tablet_manager.cc:1916] T 059716cec7f94ea6ad8989fecc6f832a P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:22.997871 28798 ts_tablet_manager.cc:1434] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20260618 13:59:23.002660 28810 raft_consensus.cc:493] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:23.002903 28810 raft_consensus.cc:515] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.003810 28810 leader_election.cc:290] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:23.004900 28484 leader_election.cc:343] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 7e5517e820ae47de810ff817f5382839
I20260618 13:59:23.002388 28220 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_03893e570e2c4d33b96437d5dbe1f241" } new_table_name: "default.Table_ddcc7a2b28ca4095be58d4c859fd964a" modify_external_catalogs: true
W20260618 13:59:23.007294 28483 leader_election.cc:343] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 7e5517e820ae47de810ff817f5382839
I20260618 13:59:23.007449 28483 leader_election.cc:304] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:23.007979 28810 raft_consensus.cc:2749] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:23.014225 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e5517e820ae47de810ff817f5382839 with delete_type TABLET_DATA_DELETED (replaced by tablet dc36fbea853a4d4e9f61909200239aed) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.021284 28549 raft_consensus.cc:3060] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:23.022087 28799 tablet_bootstrap.cc:492] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:23.022244 28799 ts_tablet_manager.cc:1403] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.180s user 0.022s sys 0.052s
I20260618 13:59:23.023054 28799 raft_consensus.cc:359] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.023233 28799 raft_consensus.cc:385] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.023329 28799 raft_consensus.cc:740] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.023562 28799 consensus_queue.cc:260] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.023856 28798 tablet_bootstrap.cc:492] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:23.024066 28799 ts_tablet_manager.cc:1434] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 13:59:23.024400 28799 tablet_bootstrap.cc:492] T 668c6c6f2034484ca5ccdd51c22dea1d P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:23.026140 28801 tablet_replica.cc:333] T d5cb3137659a4481840fe04174ef682d P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:23.026239 28799 tablet_bootstrap.cc:654] T 668c6c6f2034484ca5ccdd51c22dea1d P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.030195 28801 raft_consensus.cc:2243] T d5cb3137659a4481840fe04174ef682d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.030498 28801 raft_consensus.cc:2272] T d5cb3137659a4481840fe04174ef682d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.031222 28801 ts_tablet_manager.cc:1916] T d5cb3137659a4481840fe04174ef682d P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.033439 28799 tablet_bootstrap.cc:492] T 668c6c6f2034484ca5ccdd51c22dea1d P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:23.033603 28799 ts_tablet_manager.cc:1403] T 668c6c6f2034484ca5ccdd51c22dea1d P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.003s
I20260618 13:59:23.034386 28799 raft_consensus.cc:359] T 668c6c6f2034484ca5ccdd51c22dea1d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.034683 28799 raft_consensus.cc:385] T 668c6c6f2034484ca5ccdd51c22dea1d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.034755 28799 raft_consensus.cc:740] T 668c6c6f2034484ca5ccdd51c22dea1d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.035039 28799 consensus_queue.cc:260] T 668c6c6f2034484ca5ccdd51c22dea1d P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.035558 28799 ts_tablet_manager.cc:1434] T 668c6c6f2034484ca5ccdd51c22dea1d P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:23.035969 28799 tablet_bootstrap.cc:492] T b739fa1847184290833e13a2515f3a3a P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:23.036341 28798 tablet_bootstrap.cc:654] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.038059 28799 tablet_bootstrap.cc:654] T b739fa1847184290833e13a2515f3a3a P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.040750 28549 raft_consensus.cc:1275] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Refusing update from remote peer 6fc808af0e884ed8836561665d8e0e7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 13:59:23.043068 28225 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_9241cbda69c4474e8d4af9e391d5debb" } new_table_name: "default.Table_5d0d3ddc1f4a4badbb853268cb2bba65" modify_external_catalogs: true
I20260618 13:59:23.043643 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3dbf13544ce408ab97834aec5862390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.044260 28224 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_734fac8462fe4ad19c5d3d9a92e06f0e" } new_table_name: "default.Table_9424eec16a284672a0fe7ab3d0852b40" modify_external_catalogs: true
I20260618 13:59:23.049239 28807 raft_consensus.cc:493] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:23.049432 28807 raft_consensus.cc:515] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.050074 28807 leader_election.cc:290] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:23.050239 28807 consensus_queue.cc:1048] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:23.051520 28806 raft_consensus.cc:493] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:23.051699 28806 raft_consensus.cc:515] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.054819 28806 leader_election.cc:290] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:23.055033 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37fedd27109944e2b3c870418c117758" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:23.055675 28346 leader_election.cc:343] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:23.059293 28798 tablet_bootstrap.cc:492] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:23.059473 28798 ts_tablet_manager.cc:1403] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.042s user 0.002s sys 0.003s
I20260618 13:59:23.060226 28798 raft_consensus.cc:359] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.060706 28798 raft_consensus.cc:385] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.060784 28798 raft_consensus.cc:740] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.061036 28798 consensus_queue.cc:260] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.061189 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet bd6b0cdffd5a44b5b45b885f6e88eb9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.062883 28798 ts_tablet_manager.cc:1434] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:23.059516 28683 raft_consensus.cc:1275] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Refusing update from remote peer 6fc808af0e884ed8836561665d8e0e7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 13:59:23.064971 28798 tablet_bootstrap.cc:492] T d6e3060118e844559803a649224d7474 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:23.067538 28801 ts_tablet_manager.cc:1929] T d5cb3137659a4481840fe04174ef682d P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.067699 28801 log.cc:1199] T d5cb3137659a4481840fe04174ef682d P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/d5cb3137659a4481840fe04174ef682d
W20260618 13:59:23.068295 28617 leader_election.cc:343] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 1d23b5d2d67945a6ab9591969acb0d27
I20260618 13:59:23.059562 28648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d4144b1b5ea4e968672616bc22bead8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.068958 28934 consensus_queue.cc:1048] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:23.069458 28798 tablet_bootstrap.cc:654] T d6e3060118e844559803a649224d7474 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.071162 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d23b5d2d67945a6ab9591969acb0d27" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:23.073153 28799 tablet_bootstrap.cc:492] T b739fa1847184290833e13a2515f3a3a P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:23.073313 28799 ts_tablet_manager.cc:1403] T b739fa1847184290833e13a2515f3a3a P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.003s
I20260618 13:59:23.069746 28801 ts_tablet_manager.cc:1950] T d5cb3137659a4481840fe04174ef682d P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:23.074369 28799 raft_consensus.cc:359] T b739fa1847184290833e13a2515f3a3a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.074594 28799 raft_consensus.cc:385] T b739fa1847184290833e13a2515f3a3a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.074716 28799 raft_consensus.cc:740] T b739fa1847184290833e13a2515f3a3a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.074990 28799 consensus_queue.cc:260] T b739fa1847184290833e13a2515f3a3a P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.075559 28799 ts_tablet_manager.cc:1434] T b739fa1847184290833e13a2515f3a3a P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:23.077242 28810 raft_consensus.cc:493] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:23.077383 28810 raft_consensus.cc:515] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.077593 28801 tablet_replica.cc:333] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:23.077711 28649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 512a9e63ba324a30b8d8c37e585ebdf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.077776 28801 raft_consensus.cc:2243] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.078043 28801 raft_consensus.cc:2272] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.078071 28810 leader_election.cc:290] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:23.079644 28618 leader_election.cc:343] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:23.081094 28801 ts_tablet_manager.cc:1916] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:23.081501 28483 leader_election.cc:343] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 31b328882af549a78535cdd55c6ea09c
I20260618 13:59:23.082568 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 102a500268574e428cb2d45e52849f57 (table default.Table_bd146d3bdafe4c66b00a25d98887f1e2 [id=76865706d6a54ec198dd205b6ad61920]) successfully deleted
W20260618 13:59:23.083297 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 37a6764a5e024903848561c9935b1a43 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37a6764a5e024903848561c9935b1a43 already in progress: creating tablet
I20260618 13:59:23.075934 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37fedd27109944e2b3c870418c117758" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:23.084767 28346 leader_election.cc:343] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:23.084921 28346 leader_election.cc:304] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:23.085117 28650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98b42d23832545ae81a7cdff19e4689f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.085374 28934 raft_consensus.cc:2749] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:23.085894 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet d5cb3137659a4481840fe04174ef682d (table default.Table_c97dcabd4c2b45a18f445a099ec7ace9 [id=38bde7a301814aa285e4bd30bcc527aa]) successfully deleted
W20260618 13:59:23.086150 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d980cda1c60e4edca7b4d7537f4f5837 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d980cda1c60e4edca7b4d7537f4f5837 already in progress: creating tablet
I20260618 13:59:23.086519 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 827cc0987bd541759e334fb620f04ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.088613 28798 tablet_bootstrap.cc:492] T d6e3060118e844559803a649224d7474 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:23.088605 28799 tablet_bootstrap.cc:492] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:23.088990 28798 ts_tablet_manager.cc:1403] T d6e3060118e844559803a649224d7474 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.004s
I20260618 13:59:23.090075 28798 raft_consensus.cc:359] T d6e3060118e844559803a649224d7474 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.090623 28799 tablet_bootstrap.cc:654] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.091356 28798 raft_consensus.cc:385] T d6e3060118e844559803a649224d7474 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.091639 28798 raft_consensus.cc:740] T d6e3060118e844559803a649224d7474 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.092392 28798 consensus_queue.cc:260] T d6e3060118e844559803a649224d7474 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.095101 28798 ts_tablet_manager.cc:1434] T d6e3060118e844559803a649224d7474 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260618 13:59:23.096751 28798 tablet_bootstrap.cc:492] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:23.098814 28802 ts_tablet_manager.cc:1929] T 1f0109536c24485c9934193f2ef8aeae P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.099246 28802 log.cc:1199] T 1f0109536c24485c9934193f2ef8aeae P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/1f0109536c24485c9934193f2ef8aeae
I20260618 13:59:23.099879 28802 ts_tablet_manager.cc:1950] T 1f0109536c24485c9934193f2ef8aeae P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:23.101464 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 1f0109536c24485c9934193f2ef8aeae (table default.Table_cb42691367014b83a1ecbc738cbe0ab9 [id=cfb4ca8c4e884ccc8602be52d0b39100]) successfully deleted
I20260618 13:59:23.102626 28798 tablet_bootstrap.cc:654] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:23.103205 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet e95c88e4b0a046a3ad69ee49af47c7c3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e95c88e4b0a046a3ad69ee49af47c7c3 already in progress: creating tablet
I20260618 13:59:23.104023 28802 tablet_replica.cc:333] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:23.104594 28802 raft_consensus.cc:2243] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.105147 28801 ts_tablet_manager.cc:1929] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:23.105799 28801 log.cc:1199] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/397c7abe26f4451298558d6a0875b677
I20260618 13:59:23.106649 28801 ts_tablet_manager.cc:1950] T 397c7abe26f4451298558d6a0875b677 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:23.106652 28802 raft_consensus.cc:2272] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.084375 28618 leader_election.cc:304] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:23.107255 28802 ts_tablet_manager.cc:1916] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.107720 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b328882af549a78535cdd55c6ea09c" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:23.108053 28801 tablet_replica.cc:333] T 8c9d89fa75174a969ded07e0596db99b P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:23.108168 28806 raft_consensus.cc:2749] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:23.109892 28801 raft_consensus.cc:2243] T 8c9d89fa75174a969ded07e0596db99b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.110333 28801 raft_consensus.cc:2272] T 8c9d89fa75174a969ded07e0596db99b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.111093 28801 ts_tablet_manager.cc:1916] T 8c9d89fa75174a969ded07e0596db99b P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.109521 28798 tablet_bootstrap.cc:492] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:23.113240 28798 ts_tablet_manager.cc:1403] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.004s
I20260618 13:59:23.114080 28798 raft_consensus.cc:359] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.114838 28798 raft_consensus.cc:385] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260618 13:59:23.114909 28484 leader_election.cc:343] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:23.115245 28484 leader_election.cc:304] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:23.115653 28810 raft_consensus.cc:2749] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:23.115932 28798 raft_consensus.cc:740] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.116564 28798 consensus_queue.cc:260] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.119867 28798 ts_tablet_manager.cc:1434] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
I20260618 13:59:23.121388 28798 tablet_bootstrap.cc:492] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:23.123358 28799 tablet_bootstrap.cc:492] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:23.123557 28799 ts_tablet_manager.cc:1403] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.035s user 0.007s sys 0.005s
I20260618 13:59:23.124354 28799 raft_consensus.cc:359] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:23.124588 28799 raft_consensus.cc:385] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.124701 28799 raft_consensus.cc:740] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.125005 28799 consensus_queue.cc:260] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:23.125573 28799 ts_tablet_manager.cc:1434] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:23.126977 28858 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260618 13:59:23.127413 28799 tablet_bootstrap.cc:492] T ba0d05ea44e1473fb4b24f61eab096e4 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:23.129532 28798 tablet_bootstrap.cc:654] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.129792 28799 tablet_bootstrap.cc:654] T ba0d05ea44e1473fb4b24f61eab096e4 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.131903 28858 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 13:59:23.132279 28858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 13:59:23.132387 28934 raft_consensus.cc:493] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:23.132555 28934 raft_consensus.cc:515] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.133136 28934 leader_election.cc:290] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:23.133872 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a75029992645c5b942c0ec07862bb4" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:23.134413 28346 leader_election.cc:343] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:23.134917 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a75029992645c5b942c0ec07862bb4" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:23.135500 28346 leader_election.cc:343] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:23.135681 28346 leader_election.cc:304] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:23.135959 28934 raft_consensus.cc:2749] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:23.137337 28802 ts_tablet_manager.cc:1929] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.137548 28798 tablet_bootstrap.cc:492] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:23.138118 28802 log.cc:1199] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/37fedd27109944e2b3c870418c117758
I20260618 13:59:23.138377 28798 ts_tablet_manager.cc:1403] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.002s
I20260618 13:59:23.138849 28802 ts_tablet_manager.cc:1950] T 37fedd27109944e2b3c870418c117758 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:23.140703 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 37fedd27109944e2b3c870418c117758 (table default.Table_e9f01ff23601409189a5923e3755f4f8 [id=97b635f1072a4d82a82753d618600d4a]) successfully deleted
I20260618 13:59:23.141783 28798 raft_consensus.cc:359] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.143316 28798 raft_consensus.cc:385] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.143622 28798 raft_consensus.cc:740] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.144285 28798 consensus_queue.cc:260] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.142099 28801 ts_tablet_manager.cc:1929] T 8c9d89fa75174a969ded07e0596db99b P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.145210 28801 log.cc:1199] T 8c9d89fa75174a969ded07e0596db99b P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/8c9d89fa75174a969ded07e0596db99b
I20260618 13:59:23.145699 28801 ts_tablet_manager.cc:1950] T 8c9d89fa75174a969ded07e0596db99b P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:23.147248 28802 tablet_replica.cc:333] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:23.147495 28802 raft_consensus.cc:2243] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.147706 28802 raft_consensus.cc:2272] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.148032 28801 tablet_replica.cc:333] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:23.148380 28801 raft_consensus.cc:2243] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.148423 28799 tablet_bootstrap.cc:492] T ba0d05ea44e1473fb4b24f61eab096e4 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:23.148537 28802 ts_tablet_manager.cc:1916] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.149200 28801 raft_consensus.cc:2272] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.148905 28798 ts_tablet_manager.cc:1434] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260618 13:59:23.148567 28799 ts_tablet_manager.cc:1403] T ba0d05ea44e1473fb4b24f61eab096e4 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.002s
I20260618 13:59:23.151404 28801 ts_tablet_manager.cc:1916] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.151752 28799 raft_consensus.cc:359] T ba0d05ea44e1473fb4b24f61eab096e4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.152010 28799 raft_consensus.cc:385] T ba0d05ea44e1473fb4b24f61eab096e4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.151589 28798 tablet_bootstrap.cc:492] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:23.152114 28799 raft_consensus.cc:740] T ba0d05ea44e1473fb4b24f61eab096e4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.152735 28799 consensus_queue.cc:260] T ba0d05ea44e1473fb4b24f61eab096e4 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.153285 28799 ts_tablet_manager.cc:1434] T ba0d05ea44e1473fb4b24f61eab096e4 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:23.154145 28799 tablet_bootstrap.cc:492] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:23.156661 28798 tablet_bootstrap.cc:654] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.157904 28799 tablet_bootstrap.cc:654] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.159013 28797 tablet_bootstrap.cc:492] T 18fb8adfc8ee40e18a1da5f978317124 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:23.159240 28797 ts_tablet_manager.cc:1403] T 18fb8adfc8ee40e18a1da5f978317124 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.445s user 0.020s sys 0.021s
I20260618 13:59:23.160559 28797 raft_consensus.cc:359] T 18fb8adfc8ee40e18a1da5f978317124 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.160974 28797 raft_consensus.cc:385] T 18fb8adfc8ee40e18a1da5f978317124 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.161130 28797 raft_consensus.cc:740] T 18fb8adfc8ee40e18a1da5f978317124 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.161460 28797 consensus_queue.cc:260] T 18fb8adfc8ee40e18a1da5f978317124 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.162133 28797 ts_tablet_manager.cc:1434] T 18fb8adfc8ee40e18a1da5f978317124 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260618 13:59:23.162670 28797 tablet_bootstrap.cc:492] T 5332b6bb76ea419086341738fd135b24 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:23.164953 28802 ts_tablet_manager.cc:1929] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.165099 28802 log.cc:1199] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/bd6b0cdffd5a44b5b45b885f6e88eb9d
I20260618 13:59:23.165499 28802 ts_tablet_manager.cc:1950] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:23.167873 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet bd6b0cdffd5a44b5b45b885f6e88eb9d (table default.Table_f32d5142bcf64f5c9e94b015dd620e10 [id=cfb012a9137e481eb0e03145a6404788]) successfully deleted
I20260618 13:59:23.168246 28797 tablet_bootstrap.cc:654] T 5332b6bb76ea419086341738fd135b24 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.172122 28800 ts_tablet_manager.cc:1929] T 059716cec7f94ea6ad8989fecc6f832a P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.172295 28800 log.cc:1199] T 059716cec7f94ea6ad8989fecc6f832a P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/059716cec7f94ea6ad8989fecc6f832a
I20260618 13:59:23.172781 28800 ts_tablet_manager.cc:1950] T 059716cec7f94ea6ad8989fecc6f832a P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:23.175623 28799 tablet_bootstrap.cc:492] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:23.175820 28799 ts_tablet_manager.cc:1403] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.000s
I20260618 13:59:23.176116 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 397c7abe26f4451298558d6a0875b677 (table default.Table_2be2c24d66ae4f1d8bf1e78d134841c7 [id=ce780967c12a412a8d7f0bef2f9c00e3]) successfully deleted
I20260618 13:59:23.176426 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 8c9d89fa75174a969ded07e0596db99b (table default.Table_07cff5be28854452b80fdac31e7ecba3 [id=b024b7c54de64df0a4047d8f0381a5c8]) successfully deleted
I20260618 13:59:23.176700 28799 raft_consensus.cc:359] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.176913 28799 raft_consensus.cc:385] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.176996 28799 raft_consensus.cc:740] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.177260 28799 consensus_queue.cc:260] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.177810 28799 ts_tablet_manager.cc:1434] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:23.178164 28799 tablet_bootstrap.cc:492] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:23.178159 28801 ts_tablet_manager.cc:1929] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.178383 28801 log.cc:1199] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/b312fbf800434ecb840dd10347efe6e8
I20260618 13:59:23.178498 28797 tablet_bootstrap.cc:492] T 5332b6bb76ea419086341738fd135b24 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:23.178692 28797 ts_tablet_manager.cc:1403] T 5332b6bb76ea419086341738fd135b24 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.005s
I20260618 13:59:23.179966 28799 tablet_bootstrap.cc:654] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.179615 28798 tablet_bootstrap.cc:492] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:23.180250 28798 ts_tablet_manager.cc:1403] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.000s
I20260618 13:59:23.181603 28798 raft_consensus.cc:359] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.181859 28798 raft_consensus.cc:385] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.181988 28798 raft_consensus.cc:740] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.182402 28798 consensus_queue.cc:260] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.183995 28798 ts_tablet_manager.cc:1434] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260618 13:59:23.185945 28801 ts_tablet_manager.cc:1950] T b312fbf800434ecb840dd10347efe6e8 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:23.186517 28798 tablet_bootstrap.cc:492] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:23.187827 28797 raft_consensus.cc:359] T 5332b6bb76ea419086341738fd135b24 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.188107 28801 tablet_replica.cc:333] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:23.188119 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 059716cec7f94ea6ad8989fecc6f832a (table default.Table_e0177aee52ca4680a5786ad73020ad75 [id=591e2be369c347e883141ed55927044d]) successfully deleted
I20260618 13:59:23.188339 28797 raft_consensus.cc:385] T 5332b6bb76ea419086341738fd135b24 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.188565 28797 raft_consensus.cc:740] T 5332b6bb76ea419086341738fd135b24 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.188849 28797 consensus_queue.cc:260] T 5332b6bb76ea419086341738fd135b24 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.189613 28801 raft_consensus.cc:2243] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.189976 28797 ts_tablet_manager.cc:1434] T 5332b6bb76ea419086341738fd135b24 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
I20260618 13:59:23.190562 28797 tablet_bootstrap.cc:492] T 4ebed81a21b14db48ecb69299e0956cd P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:23.190747 28801 raft_consensus.cc:2272] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.192237 28801 ts_tablet_manager.cc:1916] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.192286 28797 tablet_bootstrap.cc:654] T 4ebed81a21b14db48ecb69299e0956cd P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.196005 29014 tablet.cc:1724] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260618 13:59:23.197546 28798 tablet_bootstrap.cc:654] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.201403 28801 ts_tablet_manager.cc:1929] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.201581 28801 log.cc:1199] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/15bbf81e3fbf4927b822afa0635909eb
I20260618 13:59:23.202054 28801 ts_tablet_manager.cc:1950] T 15bbf81e3fbf4927b822afa0635909eb P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:23.202636 28801 tablet_replica.cc:333] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:23.202888 28801 raft_consensus.cc:2243] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.203100 28801 raft_consensus.cc:2272] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.203747 28801 ts_tablet_manager.cc:1916] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.204327 28799 tablet_bootstrap.cc:492] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:23.204475 28799 ts_tablet_manager.cc:1403] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.000s
I20260618 13:59:23.205267 28799 raft_consensus.cc:359] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.205466 28799 raft_consensus.cc:385] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.205544 28799 raft_consensus.cc:740] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.205786 28799 consensus_queue.cc:260] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.210007 28799 ts_tablet_manager.cc:1434] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260618 13:59:23.210282 28799 tablet_bootstrap.cc:492] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:23.212054 28799 tablet_bootstrap.cc:654] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.219851 28226 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 29bf263d20934ed6a6efb4b215252c00 (table default.Table_5f8fbc1f56c1484cb99b55730190bbcf [id=7923680bde144e3dbb78c922d09540b9]). Expected version 4 got 0
I20260618 13:59:23.221302 28226 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 29bf263d20934ed6a6efb4b215252c00 (table default.Table_5f8fbc1f56c1484cb99b55730190bbcf [id=7923680bde144e3dbb78c922d09540b9]). Expected version 4 got 0
I20260618 13:59:23.221616 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet bd6b0cdffd5a44b5b45b885f6e88eb9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.221868 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 668c6c6f2034484ca5ccdd51c22dea1d with delete_type TABLET_DATA_DELETED (replaced by tablet 9da90366cb7442bb9f4bf3bab07db070) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.222263 28802 tablet_replica.cc:333] T 668c6c6f2034484ca5ccdd51c22dea1d P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:23.222471 28802 raft_consensus.cc:2243] T 668c6c6f2034484ca5ccdd51c22dea1d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.222663 28802 raft_consensus.cc:2272] T 668c6c6f2034484ca5ccdd51c22dea1d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:23.223060 28214 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet bd6b0cdffd5a44b5b45b885f6e88eb9d because the tablet was not found. No further retry: Not found: Tablet not found: bd6b0cdffd5a44b5b45b885f6e88eb9d
I20260618 13:59:23.223361 28802 ts_tablet_manager.cc:1916] T 668c6c6f2034484ca5ccdd51c22dea1d P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.223464 28801 ts_tablet_manager.cc:1929] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.223598 28801 log.cc:1199] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/eeffc96b9b97441d9301e4919d0047e1
I20260618 13:59:23.223949 28801 ts_tablet_manager.cc:1950] T eeffc96b9b97441d9301e4919d0047e1 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:23.226475 28801 tablet_replica.cc:333] T 85a3b27d914a412498919a3f65e20522 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:23.226686 28801 raft_consensus.cc:2243] T 85a3b27d914a412498919a3f65e20522 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.226886 28801 raft_consensus.cc:2272] T 85a3b27d914a412498919a3f65e20522 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.227414 28801 ts_tablet_manager.cc:1916] T 85a3b27d914a412498919a3f65e20522 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.229606 28802 ts_tablet_manager.cc:1929] T 668c6c6f2034484ca5ccdd51c22dea1d P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.229740 28802 log.cc:1199] T 668c6c6f2034484ca5ccdd51c22dea1d P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/668c6c6f2034484ca5ccdd51c22dea1d
I20260618 13:59:23.230103 28802 ts_tablet_manager.cc:1950] T 668c6c6f2034484ca5ccdd51c22dea1d P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:23.231796 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 668c6c6f2034484ca5ccdd51c22dea1d (table default.Table_a6146ff7daf149b4bca36d8f09293605 [id=aa7194d6b3b2419294e5aa57d0043a11]) successfully deleted
I20260618 13:59:23.233026 28994 mvcc.cc:204] Tried to move back new op lower bound from 7298216603704791040 to 7298216601900138496. Current Snapshot: MvccSnapshot[applied={T|T < 7298216603704791040}]
I20260618 13:59:23.233793 29021 tablet.cc:1724] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260618 13:59:23.235025 28799 tablet_bootstrap.cc:492] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:23.235196 28799 ts_tablet_manager.cc:1403] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.000s
I20260618 13:59:23.236057 28799 raft_consensus.cc:359] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.236706 28799 raft_consensus.cc:385] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.236791 28799 raft_consensus.cc:740] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.236687 29015 tablet.cc:1724] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260618 13:59:23.237039 28799 consensus_queue.cc:260] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.237594 28799 ts_tablet_manager.cc:1434] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:23.237919 28799 tablet_bootstrap.cc:492] T dca54111fea047d9ae0fabe376d89382 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:23.240938 28215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.62:43001, user_credentials={real_user=slave}} blocked reactor thread for 52471.3us
I20260618 13:59:23.241151 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet b312fbf800434ecb840dd10347efe6e8 (table default.Table_1ccdff414ebc4079933d0edf079a2fdb [id=7e70d6cbf42e437da0c1bb47eee6633c]) successfully deleted
I20260618 13:59:23.241340 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 15bbf81e3fbf4927b822afa0635909eb (table default.Table_1fd86f80c8fc4490aaadc1278d20df3a [id=e76fba0e75f74f8a9d0a94ff9d7f7b53]) successfully deleted
I20260618 13:59:23.241652 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet eeffc96b9b97441d9301e4919d0047e1 (table default.Table_f0607bf6529e4f209e00f49908420141 [id=972bc2c124ac459da06ec4093c5b564b]) successfully deleted
I20260618 13:59:23.243310 28798 tablet_bootstrap.cc:492] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:23.243500 28798 ts_tablet_manager.cc:1403] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.058s user 0.000s sys 0.007s
I20260618 13:59:23.244537 28798 raft_consensus.cc:359] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.244776 28798 raft_consensus.cc:385] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.244908 28798 raft_consensus.cc:740] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.245254 28798 consensus_queue.cc:260] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.245849 28798 ts_tablet_manager.cc:1434] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 13:59:23.246966 28798 tablet_bootstrap.cc:492] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:23.249862 28799 tablet_bootstrap.cc:654] T dca54111fea047d9ae0fabe376d89382 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.251044 28801 ts_tablet_manager.cc:1929] T 85a3b27d914a412498919a3f65e20522 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260618 13:59:23.255311 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 233040a16b0d4b61b977ca57c1afec2d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 233040a16b0d4b61b977ca57c1afec2d already in progress: creating tablet
I20260618 13:59:23.257095 28800 tablet_replica.cc:333] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:23.257311 28800 raft_consensus.cc:2243] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.257488 28800 raft_consensus.cc:2272] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.258002 28800 ts_tablet_manager.cc:1916] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.262360 28223 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet d7a97176f72349cda221baaea61d18ad
I20260618 13:59:23.262578 28799 tablet_bootstrap.cc:492] T dca54111fea047d9ae0fabe376d89382 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:23.262751 28799 ts_tablet_manager.cc:1403] T dca54111fea047d9ae0fabe376d89382 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.001s
I20260618 13:59:23.262810 28223 catalog_manager.cc:6784] ReplaceTablet: tablet d7a97176f72349cda221baaea61d18ad deleted and replaced by tablet 72fd63e8245d4042b091593881ba11b6
I20260618 13:59:23.263756 28799 raft_consensus.cc:359] T dca54111fea047d9ae0fabe376d89382 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.264000 28799 raft_consensus.cc:385] T dca54111fea047d9ae0fabe376d89382 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.264099 28799 raft_consensus.cc:740] T dca54111fea047d9ae0fabe376d89382 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.264240 28221 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_4cf88aaddada4361837d7d49170f029b" } modify_external_catalogs: true
I20260618 13:59:23.264428 28221 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_4cf88aaddada4361837d7d49170f029b" } modify_external_catalogs: true
I20260618 13:59:23.264380 28799 consensus_queue.cc:260] T dca54111fea047d9ae0fabe376d89382 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.265015 28799 ts_tablet_manager.cc:1434] T dca54111fea047d9ae0fabe376d89382 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:23.265358 28799 tablet_bootstrap.cc:492] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:23.266495 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d6b01b897134d5bbd6a7d93b82ddb5e already in progress: creating tablet
W20260618 13:59:23.266775 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet dca54111fea047d9ae0fabe376d89382 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dca54111fea047d9ae0fabe376d89382 already in progress: creating tablet
I20260618 13:59:23.270259 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet d7a97176f72349cda221baaea61d18ad with delete_type TABLET_DATA_DELETED (replaced by tablet 72fd63e8245d4042b091593881ba11b6) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.274144 28797 tablet_bootstrap.cc:492] T 4ebed81a21b14db48ecb69299e0956cd P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:23.274286 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet d7a97176f72349cda221baaea61d18ad with delete_type TABLET_DATA_DELETED (replaced by tablet 72fd63e8245d4042b091593881ba11b6) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.275106 28797 ts_tablet_manager.cc:1403] T 4ebed81a21b14db48ecb69299e0956cd P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.085s user 0.003s sys 0.016s
I20260618 13:59:23.275139 28799 tablet_bootstrap.cc:654] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:23.275604 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet d7a97176f72349cda221baaea61d18ad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d7a97176f72349cda221baaea61d18ad already in progress: creating tablet
I20260618 13:59:23.275895 28797 raft_consensus.cc:359] T 4ebed81a21b14db48ecb69299e0956cd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.276089 28797 raft_consensus.cc:385] T 4ebed81a21b14db48ecb69299e0956cd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.276180 28797 raft_consensus.cc:740] T 4ebed81a21b14db48ecb69299e0956cd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.276463 28797 consensus_queue.cc:260] T 4ebed81a21b14db48ecb69299e0956cd P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.278095 28797 ts_tablet_manager.cc:1434] T 4ebed81a21b14db48ecb69299e0956cd P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:23.279177 28797 tablet_bootstrap.cc:492] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:23.281695 28797 tablet_bootstrap.cc:654] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.282205 28798 tablet_bootstrap.cc:654] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.282173 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet d7a97176f72349cda221baaea61d18ad with delete_type TABLET_DATA_DELETED (replaced by tablet 72fd63e8245d4042b091593881ba11b6) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.282142 28801 log.cc:1199] T 85a3b27d914a412498919a3f65e20522 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/85a3b27d914a412498919a3f65e20522
I20260618 13:59:23.284106 28801 ts_tablet_manager.cc:1950] T 85a3b27d914a412498919a3f65e20522 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:23.284441 28798 tablet_bootstrap.cc:492] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:23.284737 28798 ts_tablet_manager.cc:1403] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.038s user 0.003s sys 0.001s
I20260618 13:59:23.285651 28798 raft_consensus.cc:359] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.285918 28798 raft_consensus.cc:385] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.286010 28798 raft_consensus.cc:740] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.286383 28798 consensus_queue.cc:260] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.288659 28798 ts_tablet_manager.cc:1434] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:23.288975 28798 tablet_bootstrap.cc:492] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:23.290223 28799 tablet_bootstrap.cc:492] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:23.290597 28799 ts_tablet_manager.cc:1403] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.006s
I20260618 13:59:23.290832 28798 tablet_bootstrap.cc:654] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.291548 28799 raft_consensus.cc:359] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.291853 28799 raft_consensus.cc:385] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.291981 28799 raft_consensus.cc:740] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.292402 28799 consensus_queue.cc:260] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.294057 28799 ts_tablet_manager.cc:1434] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260618 13:59:23.294265 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 85a3b27d914a412498919a3f65e20522 (table default.Table_94d9d315c21e4cfb9950d96262c9b6e3 [id=a874707bba8d41e7b598d7ad4a3e21f3]) successfully deleted
I20260618 13:59:23.295258 28799 tablet_bootstrap.cc:492] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:23.296082 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 233040a16b0d4b61b977ca57c1afec2d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 233040a16b0d4b61b977ca57c1afec2d already in progress: creating tablet
I20260618 13:59:23.297253 28799 tablet_bootstrap.cc:654] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:23.297209 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d6b01b897134d5bbd6a7d93b82ddb5e already in progress: creating tablet
W20260618 13:59:23.298189 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet dca54111fea047d9ae0fabe376d89382 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dca54111fea047d9ae0fabe376d89382 already in progress: creating tablet
W20260618 13:59:23.299177 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 94365c44b5e24acab5df60c67203794a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 94365c44b5e24acab5df60c67203794a already in progress: creating tablet
W20260618 13:59:23.300263 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 311ba4dd83984f379c8f4f8274bc26c1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 311ba4dd83984f379c8f4f8274bc26c1 already in progress: creating tablet
W20260618 13:59:23.301146 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet e025b64380314546af8a1ab90c817825 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e025b64380314546af8a1ab90c817825 already in progress: creating tablet
W20260618 13:59:23.302105 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 2188ebf1727240cd886ba7662f8fcf43 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2188ebf1727240cd886ba7662f8fcf43 already in progress: creating tablet
W20260618 13:59:23.303073 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 1f0109536c24485c9934193f2ef8aeae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f0109536c24485c9934193f2ef8aeae already in progress: creating tablet
W20260618 13:59:23.304329 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet e95c88e4b0a046a3ad69ee49af47c7c3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e95c88e4b0a046a3ad69ee49af47c7c3 already in progress: creating tablet
W20260618 13:59:23.305814 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 37fedd27109944e2b3c870418c117758 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37fedd27109944e2b3c870418c117758 already in progress: creating tablet
I20260618 13:59:23.306234 28800 ts_tablet_manager.cc:1929] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.306366 28800 log.cc:1199] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/1d23b5d2d67945a6ab9591969acb0d27
I20260618 13:59:23.306787 28801 tablet_replica.cc:333] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:23.307051 28801 raft_consensus.cc:2243] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.307282 28801 raft_consensus.cc:2272] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.308856 28798 tablet_bootstrap.cc:492] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:23.309046 28798 ts_tablet_manager.cc:1403] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.000s
I20260618 13:59:23.309885 28798 raft_consensus.cc:359] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.310103 28798 raft_consensus.cc:385] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.310215 28798 raft_consensus.cc:740] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.311069 28801 ts_tablet_manager.cc:1916] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.314754 28798 consensus_queue.cc:260] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.315467 28798 ts_tablet_manager.cc:1434] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260618 13:59:23.315909 28798 tablet_bootstrap.cc:492] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:23.317732 28798 tablet_bootstrap.cc:654] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:23.323140 28214 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet a1a06078974e4bfbbc4b949802a39bf1 (table default.Table_4de8ba099efe4bb3bc0a0d90e1861107 [id=7b747ef3b2494ca78c499c5bb75648d4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a1a06078974e4bfbbc4b949802a39bf1 consensus state has no leader
I20260618 13:59:23.323798 28800 ts_tablet_manager.cc:1950] T 1d23b5d2d67945a6ab9591969acb0d27 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
W20260618 13:59:23.324136 28227 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 69d5e75b4a8247b29d69d07fad5ca5db consensus state has no leader
I20260618 13:59:23.324872 28222 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 751cd9bd961b4f56a588ed02a05f0212
W20260618 13:59:23.325560 28220 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b0c9d4131ea24809ac081e7a73a21e1c consensus state has no leader
I20260618 13:59:23.325825 28220 master_service.cc:960] ReplaceTablet: received request to replace tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:23.326117 28228 master_service.cc:960] ReplaceTablet: received request to replace tablet 092a9aba9c6e48f5be9cf5d55ba93678 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:23.329584 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 751cd9bd961b4f56a588ed02a05f0212 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:23.330549 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 751cd9bd961b4f56a588ed02a05f0212 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 751cd9bd961b4f56a588ed02a05f0212 already in progress: creating tablet
I20260618 13:59:23.331496 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 1d23b5d2d67945a6ab9591969acb0d27 (table default.Table_48dff8bdaa4e430b8850f8d688195bcf [id=fa4cd457fb154247acbfc842852eef04]) successfully deleted
I20260618 13:59:23.331537 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet 751cd9bd961b4f56a588ed02a05f0212 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43976
W20260618 13:59:23.332360 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 311ba4dd83984f379c8f4f8274bc26c1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 311ba4dd83984f379c8f4f8274bc26c1 already in progress: creating tablet
I20260618 13:59:23.332892 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet 751cd9bd961b4f56a588ed02a05f0212 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.333190 28934 raft_consensus.cc:493] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260618 13:59:23.333330 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet e025b64380314546af8a1ab90c817825 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e025b64380314546af8a1ab90c817825 already in progress: creating tablet
I20260618 13:59:23.333367 28934 raft_consensus.cc:515] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:23.333737 28797 tablet_bootstrap.cc:492] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:23.333909 28797 ts_tablet_manager.cc:1403] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.056s user 0.009s sys 0.001s
W20260618 13:59:23.334004 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 2188ebf1727240cd886ba7662f8fcf43 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2188ebf1727240cd886ba7662f8fcf43 already in progress: creating tablet
I20260618 13:59:23.334095 28934 leader_election.cc:290] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:23.334682 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 1f0109536c24485c9934193f2ef8aeae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f0109536c24485c9934193f2ef8aeae already in progress: creating tablet
I20260618 13:59:23.335044 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f3dc9c3d874ca4ae42fd5e6d84fcb0" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:23.335067 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f3dc9c3d874ca4ae42fd5e6d84fcb0" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:23.335378 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 94365c44b5e24acab5df60c67203794a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 94365c44b5e24acab5df60c67203794a already in progress: creating tablet
W20260618 13:59:23.335590 28346 leader_election.cc:343] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:23.335875 28346 leader_election.cc:343] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:23.335915 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet e95c88e4b0a046a3ad69ee49af47c7c3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e95c88e4b0a046a3ad69ee49af47c7c3 already in progress: creating tablet
I20260618 13:59:23.336014 28346 leader_election.cc:304] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:23.336508 28934 raft_consensus.cc:2749] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:23.336547 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 37fedd27109944e2b3c870418c117758 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37fedd27109944e2b3c870418c117758 already in progress: creating tablet
W20260618 13:59:23.337086 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d7a97176f72349cda221baaea61d18ad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d7a97176f72349cda221baaea61d18ad already in progress: creating tablet
I20260618 13:59:23.336990 28797 raft_consensus.cc:359] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.337379 28797 raft_consensus.cc:385] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.337519 28797 raft_consensus.cc:740] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
W20260618 13:59:23.337682 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 751cd9bd961b4f56a588ed02a05f0212 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 751cd9bd961b4f56a588ed02a05f0212 already in progress: creating tablet
I20260618 13:59:23.337939 28797 consensus_queue.cc:260] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.340343 28799 tablet_bootstrap.cc:492] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:23.341334 28797 ts_tablet_manager.cc:1434] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260618 13:59:23.341416 28799 ts_tablet_manager.cc:1403] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.047s user 0.003s sys 0.001s
I20260618 13:59:23.342715 28797 tablet_bootstrap.cc:492] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:23.344393 28799 raft_consensus.cc:359] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.344614 28799 raft_consensus.cc:385] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.344718 28799 raft_consensus.cc:740] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.344970 28799 consensus_queue.cc:260] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.346076 28801 ts_tablet_manager.cc:1929] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.347394 28797 tablet_bootstrap.cc:654] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.348124 28801 log.cc:1199] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/7e5517e820ae47de810ff817f5382839
I20260618 13:59:23.349783 28801 ts_tablet_manager.cc:1950] T 7e5517e820ae47de810ff817f5382839 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
W20260618 13:59:23.351420 28224 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b6dd422c05324c989e29b20ae2afabad consensus state has no leader
I20260618 13:59:23.352836 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_7d317fa28e3142a19d15c8360151c357"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:23.353483 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7d317fa28e3142a19d15c8360151c357 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.353999 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_726d3825b282496fb87c67a8975e3fcc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:23.354609 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_726d3825b282496fb87c67a8975e3fcc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.354959 28222 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_5f8fbc1f56c1484cb99b55730190bbcf" } modify_external_catalogs: true
I20260618 13:59:23.355141 28222 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_5f8fbc1f56c1484cb99b55730190bbcf" } modify_external_catalogs: true
I20260618 13:59:23.356235 28221 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet fca76774f119435899188313a33dad98
I20260618 13:59:23.356333 28799 ts_tablet_manager.cc:1434] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.015s user 0.001s sys 0.001s
I20260618 13:59:23.357067 28799 tablet_bootstrap.cc:492] T 8be4374ae7cf4907940ae90299bf9ba0 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:23.357043 28221 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_4e9ed3af75b34b83bca411019597566d" } new_table_name: "default.Table_f8635c3fdf8b4587bf0ac7791405eeac" modify_external_catalogs: true
I20260618 13:59:23.358845 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet fca76774f119435899188313a33dad98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.358876 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet fca76774f119435899188313a33dad98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:23.359431 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet fca76774f119435899188313a33dad98 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fca76774f119435899188313a33dad98 already in progress: creating tablet
I20260618 13:59:23.360661 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet fca76774f119435899188313a33dad98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:23.362011 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet fca76774f119435899188313a33dad98 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fca76774f119435899188313a33dad98 already in progress: creating tablet
I20260618 13:59:23.366173 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_74fc7c21dc664ff8bfe0d25aee399bdb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:23.366608 28227 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_fe8f6240599e467a92887643cc9d210b" } modify_external_catalogs: true
I20260618 13:59:23.366824 28227 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_fe8f6240599e467a92887643cc9d210b" } modify_external_catalogs: true
W20260618 13:59:23.367748 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_74fc7c21dc664ff8bfe0d25aee399bdb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.369143 28934 raft_consensus.cc:493] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:23.369181 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 7e5517e820ae47de810ff817f5382839 (table default.Table_4e9ed3af75b34b83bca411019597566d [id=49576dabdecf41aca1b15ff3fc0c113c]) successfully deleted
I20260618 13:59:23.369333 28934 raft_consensus.cc:515] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.370033 28934 leader_election.cc:290] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:23.370874 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95db5ddaf5d5486988750efed5972eb6" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:23.371594 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95db5ddaf5d5486988750efed5972eb6" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:23.372502 28346 leader_election.cc:343] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:23.372740 28346 leader_election.cc:343] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:23.372869 28346 leader_election.cc:304] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:23.373371 28934 raft_consensus.cc:2749] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:23.374284 28221 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dc36fbea853a4d4e9f61909200239aed consensus state has no leader
I20260618 13:59:23.374724 28221 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_d16c53f18154437e8d06e88da66faf13" } modify_external_catalogs: true
I20260618 13:59:23.374888 28221 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_d16c53f18154437e8d06e88da66faf13" } modify_external_catalogs: true
I20260618 13:59:23.376734 28798 tablet_bootstrap.cc:492] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:23.377646 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_7bdb43ababee47dc9ea2c9ec755fdbce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:23.378332 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7bdb43ababee47dc9ea2c9ec755fdbce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.379737 28229 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 29bf263d20934ed6a6efb4b215252c00 (table default.Table_5f8fbc1f56c1484cb99b55730190bbcf [id=7923680bde144e3dbb78c922d09540b9]). Expected version 4 got 0
I20260618 13:59:23.380025 28229 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 411b0c333ae7450f9ada90afee553615 (table default.Table_9241cbda69c4474e8d4af9e391d5debb [id=cd89063e7d294d3fbb3b58358e781c90]). Expected version 1 got 0
W20260618 13:59:23.380697 28225 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 411b0c333ae7450f9ada90afee553615 consensus state has no leader
I20260618 13:59:23.381021 28225 master_service.cc:960] ReplaceTablet: received request to replace tablet 4ebed81a21b14db48ecb69299e0956cd from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:23.381563 28223 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 29bf263d20934ed6a6efb4b215252c00 (table default.Table_5f8fbc1f56c1484cb99b55730190bbcf [id=7923680bde144e3dbb78c922d09540b9]). Expected version 4 got 0
I20260618 13:59:23.382100 28226 catalog_manager.cc:6241] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: default.Table_5f8fbc1f56c1484cb99b55730190bbcf [id=7923680bde144e3dbb78c922d09540b9] alter complete (version 4)
I20260618 13:59:23.382225 28799 tablet_bootstrap.cc:654] T 8be4374ae7cf4907940ae90299bf9ba0 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.383642 28226 master_service.cc:960] ReplaceTablet: received request to replace tablet b16f7d3c7837498e996bba1818110642 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:23.386873 28388 tablet_service.cc:1558] Processing DeleteTablet for tablet b739fa1847184290833e13a2515f3a3a with delete_type TABLET_DATA_DELETED (replaced by tablet 36b8a46a41e543c58a946b6764618c7d) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.386922 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 with delete_type TABLET_DATA_DELETED (replaced by tablet ac58e939730c4af681c2f08bcfbb6602) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.387182 28802 tablet_replica.cc:333] T b739fa1847184290833e13a2515f3a3a P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:23.387231 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet ba0d05ea44e1473fb4b24f61eab096e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.387419 28802 raft_consensus.cc:2243] T b739fa1847184290833e13a2515f3a3a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.387596 28802 raft_consensus.cc:2272] T b739fa1847184290833e13a2515f3a3a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.387178 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 95db5ddaf5d5486988750efed5972eb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.388357 28802 ts_tablet_manager.cc:1916] T b739fa1847184290833e13a2515f3a3a P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.391606 28656 tablet_service.cc:1511] Processing CreateTablet for tablet 2f3d2f236fd44ab8af0524ea282cfa94 (DEFAULT_TABLE table=default.Table_cd647e83c17a49219bc44d240cb9fb66 [id=e8a3305c5ad142e98b34b6bba79c9735]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.392021 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f3d2f236fd44ab8af0524ea282cfa94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.394205 28802 ts_tablet_manager.cc:1929] T b739fa1847184290833e13a2515f3a3a P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.394461 28802 log.cc:1199] T b739fa1847184290833e13a2515f3a3a P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/b739fa1847184290833e13a2515f3a3a
I20260618 13:59:23.394572 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 2f3d2f236fd44ab8af0524ea282cfa94 (DEFAULT_TABLE table=default.Table_cd647e83c17a49219bc44d240cb9fb66 [id=e8a3305c5ad142e98b34b6bba79c9735]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.394850 28802 ts_tablet_manager.cc:1950] T b739fa1847184290833e13a2515f3a3a P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:23.395141 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f3d2f236fd44ab8af0524ea282cfa94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.396353 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet b739fa1847184290833e13a2515f3a3a (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]) successfully deleted
I20260618 13:59:23.396718 28802 tablet_replica.cc:333] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:23.396914 28802 raft_consensus.cc:2243] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.397092 28802 raft_consensus.cc:2272] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.397816 28802 ts_tablet_manager.cc:1916] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.402398 28228 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 092a9aba9c6e48f5be9cf5d55ba93678
I20260618 13:59:23.402947 28228 catalog_manager.cc:6784] ReplaceTablet: tablet 092a9aba9c6e48f5be9cf5d55ba93678 deleted and replaced by tablet c7f42997416841a7b2c1ad347d2e9282
I20260618 13:59:23.404160 28659 tablet_service.cc:1558] Processing DeleteTablet for tablet 092a9aba9c6e48f5be9cf5d55ba93678 with delete_type TABLET_DATA_DELETED (replaced by tablet c7f42997416841a7b2c1ad347d2e9282) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.405182 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet 092a9aba9c6e48f5be9cf5d55ba93678 with delete_type TABLET_DATA_DELETED (replaced by tablet c7f42997416841a7b2c1ad347d2e9282) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:23.405241 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 092a9aba9c6e48f5be9cf5d55ba93678 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 092a9aba9c6e48f5be9cf5d55ba93678 already in progress: creating tablet
I20260618 13:59:23.405637 28221 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 283523e5b22843ba935eabe6283d3a99
I20260618 13:59:23.405886 28659 tablet_service.cc:1511] Processing CreateTablet for tablet ece7909d056c4ba8a5dca6423f0c1781 (DEFAULT_TABLE table=default.Table_c940d6e3d5d34adfaf39c7b5b60044e1 [id=803d5220ce1a4dcfaf27d93c574be2ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.406282 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ece7909d056c4ba8a5dca6423f0c1781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.406447 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_93b6c79a288040ceae924af14f939c35"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:23.406567 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 118259402d7d48dda381cdd18eec0f1b (DEFAULT_TABLE table=default.Table_7cea29656b76480fb24d915e7478121a [id=41e4176c7e17448a87ba484c4b447fe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.406946 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 118259402d7d48dda381cdd18eec0f1b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:23.407049 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93b6c79a288040ceae924af14f939c35 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.406092 28802 ts_tablet_manager.cc:1929] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.407527 28802 log.cc:1199] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/b1f3dc9c3d874ca4ae42fd5e6d84fcb0
I20260618 13:59:23.408427 28802 ts_tablet_manager.cc:1950] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:23.409047 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet 283523e5b22843ba935eabe6283d3a99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.409471 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 (table default.Table_ceb33d60d86e4fd898ae67c78da94a55 [id=3251ae257e0f49b99352dcc1ef004588]) successfully deleted
I20260618 13:59:23.410485 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet 283523e5b22843ba935eabe6283d3a99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.410676 28802 tablet_replica.cc:333] T ba0d05ea44e1473fb4b24f61eab096e4 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:23.410884 28802 raft_consensus.cc:2243] T ba0d05ea44e1473fb4b24f61eab096e4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.411026 28802 raft_consensus.cc:2272] T ba0d05ea44e1473fb4b24f61eab096e4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.412492 28802 ts_tablet_manager.cc:1916] T ba0d05ea44e1473fb4b24f61eab096e4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:23.413470 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 283523e5b22843ba935eabe6283d3a99 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 283523e5b22843ba935eabe6283d3a99 already in progress: creating tablet
I20260618 13:59:23.414407 28656 tablet_service.cc:1511] Processing CreateTablet for tablet 118259402d7d48dda381cdd18eec0f1b (DEFAULT_TABLE table=default.Table_7cea29656b76480fb24d915e7478121a [id=41e4176c7e17448a87ba484c4b447fe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.414956 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 118259402d7d48dda381cdd18eec0f1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.417358 28798 ts_tablet_manager.cc:1403] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.102s user 0.002s sys 0.002s
I20260618 13:59:23.418345 28798 raft_consensus.cc:359] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.418619 28798 raft_consensus.cc:385] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.418717 28798 raft_consensus.cc:740] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.419180 28220 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9
I20260618 13:59:23.419708 28220 catalog_manager.cc:6784] ReplaceTablet: tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 deleted and replaced by tablet cea4bc13eabc4dd98b4f96723498014c
I20260618 13:59:23.419873 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_41e8e8de10ce4085b62fe5dbb7b4dcdc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:23.420135 28659 tablet_service.cc:1558] Processing DeleteTablet for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 with delete_type TABLET_DATA_DELETED (replaced by tablet cea4bc13eabc4dd98b4f96723498014c) from {username='slave'} at 127.0.0.1:43976
W20260618 13:59:23.420496 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_41e8e8de10ce4085b62fe5dbb7b4dcdc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.420671 28802 ts_tablet_manager.cc:1929] T ba0d05ea44e1473fb4b24f61eab096e4 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.421205 28802 log.cc:1199] T ba0d05ea44e1473fb4b24f61eab096e4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/ba0d05ea44e1473fb4b24f61eab096e4
I20260618 13:59:23.421326 28525 tablet_service.cc:1558] Processing DeleteTablet for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 with delete_type TABLET_DATA_DELETED (replaced by tablet cea4bc13eabc4dd98b4f96723498014c) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:23.421141 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 already in progress: creating tablet
I20260618 13:59:23.421578 28802 ts_tablet_manager.cc:1950] T ba0d05ea44e1473fb4b24f61eab096e4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:23.421461 28798 consensus_queue.cc:260] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.399495 28519 tablet_service.cc:1511] Processing CreateTablet for tablet 2f3d2f236fd44ab8af0524ea282cfa94 (DEFAULT_TABLE table=default.Table_cd647e83c17a49219bc44d240cb9fb66 [id=e8a3305c5ad142e98b34b6bba79c9735]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.421985 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 with delete_type TABLET_DATA_DELETED (replaced by tablet cea4bc13eabc4dd98b4f96723498014c) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.422290 28519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f3d2f236fd44ab8af0524ea282cfa94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.403954 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 092a9aba9c6e48f5be9cf5d55ba93678 with delete_type TABLET_DATA_DELETED (replaced by tablet c7f42997416841a7b2c1ad347d2e9282) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.405675 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 118259402d7d48dda381cdd18eec0f1b (DEFAULT_TABLE table=default.Table_7cea29656b76480fb24d915e7478121a [id=41e4176c7e17448a87ba484c4b447fe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.406096 28524 tablet_service.cc:1511] Processing CreateTablet for tablet ece7909d056c4ba8a5dca6423f0c1781 (DEFAULT_TABLE table=default.Table_c940d6e3d5d34adfaf39c7b5b60044e1 [id=803d5220ce1a4dcfaf27d93c574be2ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.422979 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 118259402d7d48dda381cdd18eec0f1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.423188 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ece7909d056c4ba8a5dca6423f0c1781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.413002 28810 raft_consensus.cc:493] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:23.413235 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet 283523e5b22843ba935eabe6283d3a99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.424425 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet ba0d05ea44e1473fb4b24f61eab096e4 (table default.Table_e9839256914f48e6bb6480c5db7eb379 [id=7d40d815ea4544aea591ca23b96b5dd1]) successfully deleted
I20260618 13:59:23.424459 28806 raft_consensus.cc:493] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:23.424624 28806 raft_consensus.cc:515] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.425479 28806 leader_election.cc:290] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:23.426326 28617 leader_election.cc:343] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 31b328882af549a78535cdd55c6ea09c
I20260618 13:59:23.427168 28225 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 4ebed81a21b14db48ecb69299e0956cd
I20260618 13:59:23.427742 28225 catalog_manager.cc:6784] ReplaceTablet: tablet 4ebed81a21b14db48ecb69299e0956cd deleted and replaced by tablet 6eea4d59935d40aa97dc10ec7469aa56
I20260618 13:59:23.428037 28388 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ebed81a21b14db48ecb69299e0956cd with delete_type TABLET_DATA_DELETED (replaced by tablet 6eea4d59935d40aa97dc10ec7469aa56) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.428126 28802 tablet_replica.cc:333] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:23.428138 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ebed81a21b14db48ecb69299e0956cd with delete_type TABLET_DATA_DELETED (replaced by tablet 6eea4d59935d40aa97dc10ec7469aa56) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.428242 28659 tablet_service.cc:1511] Processing CreateTablet for tablet 6b8f273def3e4b7caa4d624cf87a106c (DEFAULT_TABLE table=default.Table_cb42691367014b83a1ecbc738cbe0ab9 [id=cfb4ca8c4e884ccc8602be52d0b39100]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.428334 28802 raft_consensus.cc:2243] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.428402 28800 tablet_replica.cc:333] T 4ebed81a21b14db48ecb69299e0956cd P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:23.428515 28802 raft_consensus.cc:2272] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.428493 28385 tablet_service.cc:1511] Processing CreateTablet for tablet 6b8f273def3e4b7caa4d624cf87a106c (DEFAULT_TABLE table=default.Table_cb42691367014b83a1ecbc738cbe0ab9 [id=cfb4ca8c4e884ccc8602be52d0b39100]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.428615 28800 raft_consensus.cc:2243] T 4ebed81a21b14db48ecb69299e0956cd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.428820 28800 raft_consensus.cc:2272] T 4ebed81a21b14db48ecb69299e0956cd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.428879 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b8f273def3e4b7caa4d624cf87a106c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.429636 28800 ts_tablet_manager.cc:1916] T 4ebed81a21b14db48ecb69299e0956cd P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.429765 28802 ts_tablet_manager.cc:1916] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.428624 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b8f273def3e4b7caa4d624cf87a106c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.434113 28389 tablet_service.cc:1511] Processing CreateTablet for tablet ece7909d056c4ba8a5dca6423f0c1781 (DEFAULT_TABLE table=default.Table_c940d6e3d5d34adfaf39c7b5b60044e1 [id=803d5220ce1a4dcfaf27d93c574be2ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.434561 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ece7909d056c4ba8a5dca6423f0c1781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.434774 28388 tablet_service.cc:1511] Processing CreateTablet for tablet c0a42aa779ae452e9420d730bf8d7d55 (DEFAULT_TABLE table=default.Table_fac3fdd9b466497683bc00300a85fed1 [id=a4ac38531af34b4681abba96b42ffb7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.435132 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0a42aa779ae452e9420d730bf8d7d55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.435149 28650 tablet_service.cc:1511] Processing CreateTablet for tablet c0a42aa779ae452e9420d730bf8d7d55 (DEFAULT_TABLE table=default.Table_fac3fdd9b466497683bc00300a85fed1 [id=a4ac38531af34b4681abba96b42ffb7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.435959 28650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0a42aa779ae452e9420d730bf8d7d55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.437423 28802 ts_tablet_manager.cc:1929] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.437529 28802 log.cc:1199] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/95db5ddaf5d5486988750efed5972eb6
I20260618 13:59:23.437882 28802 ts_tablet_manager.cc:1950] T 95db5ddaf5d5486988750efed5972eb6 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:23.438218 28800 ts_tablet_manager.cc:1929] T 4ebed81a21b14db48ecb69299e0956cd P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.438328 28800 log.cc:1199] T 4ebed81a21b14db48ecb69299e0956cd P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/4ebed81a21b14db48ecb69299e0956cd
I20260618 13:59:23.439374 28228 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b0c9d4131ea24809ac081e7a73a21e1c (table default.Table_ddcc7a2b28ca4095be58d4c859fd964a [id=0bd492e029e346f68f2db501d798ace1]). Expected version 2 got 0
I20260618 13:59:23.439750 28228 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3752f49c9fc64071a45e0ce4ff4cfef2 (table default.Table_046616bed79749d5923cbd9ab9dff2b4 [id=af7cd415b54c4e4d8c7fb6f29dc8ff7b]). Expected version 1 got 0
I20260618 13:59:23.439891 28800 ts_tablet_manager.cc:1950] T 4ebed81a21b14db48ecb69299e0956cd P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:23.440361 28222 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 29bf263d20934ed6a6efb4b215252c00
I20260618 13:59:23.440326 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b328882af549a78535cdd55c6ea09c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:23.440562 28548 raft_consensus.cc:2468] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:23.441084 28618 leader_election.cc:304] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:23.441576 28806 raft_consensus.cc:2804] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:23.441712 28806 raft_consensus.cc:493] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:23.441823 28806 raft_consensus.cc:3060] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:23.441854 28649 tablet_service.cc:1558] Processing DeleteTablet for tablet 29bf263d20934ed6a6efb4b215252c00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.443584 28648 tablet_service.cc:1558] Processing DeleteTablet for tablet 18fb8adfc8ee40e18a1da5f978317124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.443874 28649 tablet_service.cc:1558] Processing DeleteTablet for tablet 5332b6bb76ea419086341738fd135b24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.446187 28802 tablet_replica.cc:333] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:23.446434 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 29bf263d20934ed6a6efb4b215252c00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.446527 28382 tablet_service.cc:1558] Processing DeleteTablet for tablet dca54111fea047d9ae0fabe376d89382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.446449 28802 raft_consensus.cc:2243] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.447675 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 95db5ddaf5d5486988750efed5972eb6 (table default.Table_067483ebd76c49e595543cc25ff1db56 [id=5dadba9b472745259796cf9015e88c65]) successfully deleted
I20260618 13:59:23.447744 28802 raft_consensus.cc:2272] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:23.447958 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 092a9aba9c6e48f5be9cf5d55ba93678 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 092a9aba9c6e48f5be9cf5d55ba93678 already in progress: creating tablet
I20260618 13:59:23.448288 28802 ts_tablet_manager.cc:1916] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.449523 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_d31dc25d118d4cd2bb4dd8f034752827"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:23.449987 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet 29bf263d20934ed6a6efb4b215252c00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:23.450297 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d31dc25d118d4cd2bb4dd8f034752827 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:23.450541 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 283523e5b22843ba935eabe6283d3a99 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 283523e5b22843ba935eabe6283d3a99 already in progress: creating tablet
W20260618 13:59:23.450806 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 already in progress: creating tablet
I20260618 13:59:23.452108 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ebed81a21b14db48ecb69299e0956cd with delete_type TABLET_DATA_DELETED (replaced by tablet 6eea4d59935d40aa97dc10ec7469aa56) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.451680 28810 raft_consensus.cc:515] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.452917 28525 tablet_service.cc:1511] Processing CreateTablet for tablet c0a42aa779ae452e9420d730bf8d7d55 (DEFAULT_TABLE table=default.Table_fac3fdd9b466497683bc00300a85fed1 [id=a4ac38531af34b4681abba96b42ffb7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.453393 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0a42aa779ae452e9420d730bf8d7d55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.456014 28802 ts_tablet_manager.cc:1929] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.456161 28802 log.cc:1199] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/4ebed81a21b14db48ecb69299e0956cd
I20260618 13:59:23.456523 28802 ts_tablet_manager.cc:1950] T 4ebed81a21b14db48ecb69299e0956cd P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:23.456912 28800 tablet_replica.cc:333] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:23.457154 28800 raft_consensus.cc:2243] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.457376 28800 raft_consensus.cc:2272] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.457671 28802 tablet_replica.cc:333] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:23.457854 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 4ebed81a21b14db48ecb69299e0956cd (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]) successfully deleted
I20260618 13:59:23.457906 28802 raft_consensus.cc:2243] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:23.458184 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet 31b328882af549a78535cdd55c6ea09c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.458458 28802 raft_consensus.cc:2272] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.459282 28802 ts_tablet_manager.cc:1916] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.461098 28806 raft_consensus.cc:515] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:23.465121 28617 leader_election.cc:343] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 31b328882af549a78535cdd55c6ea09c
I20260618 13:59:23.465453 28226 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet b16f7d3c7837498e996bba1818110642
I20260618 13:59:23.465574 28802 ts_tablet_manager.cc:1929] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:23.465716 28802 log.cc:1199] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/29bf263d20934ed6a6efb4b215252c00
I20260618 13:59:23.466105 28226 catalog_manager.cc:6784] ReplaceTablet: tablet b16f7d3c7837498e996bba1818110642 deleted and replaced by tablet d1baf86497d442dbbbddbc3498bebbee
I20260618 13:59:23.466109 28802 ts_tablet_manager.cc:1950] T 29bf263d20934ed6a6efb4b215252c00 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:23.466233 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet b16f7d3c7837498e996bba1818110642 with delete_type TABLET_DATA_DELETED (replaced by tablet d1baf86497d442dbbbddbc3498bebbee) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.466663 28227 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 864f2602840c40fc8606b3784e671cba
I20260618 13:59:23.466744 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet 31b328882af549a78535cdd55c6ea09c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.467007 28802 tablet_replica.cc:333] T dca54111fea047d9ae0fabe376d89382 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:23.467180 28800 ts_tablet_manager.cc:1916] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.467283 28802 raft_consensus.cc:2243] T dca54111fea047d9ae0fabe376d89382 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.467484 28802 raft_consensus.cc:2272] T dca54111fea047d9ae0fabe376d89382 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.468464 28802 ts_tablet_manager.cc:1916] T dca54111fea047d9ae0fabe376d89382 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.472325 28806 leader_election.cc:290] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:23.473663 28659 tablet_service.cc:1558] Processing DeleteTablet for tablet 864f2602840c40fc8606b3784e671cba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.473675 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ebed81a21b14db48ecb69299e0956cd with delete_type TABLET_DATA_DELETED (replaced by tablet 6eea4d59935d40aa97dc10ec7469aa56) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.476177 28802 ts_tablet_manager.cc:1929] T dca54111fea047d9ae0fabe376d89382 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.476323 28802 log.cc:1199] T dca54111fea047d9ae0fabe376d89382 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/dca54111fea047d9ae0fabe376d89382
I20260618 13:59:23.476318 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 4ebed81a21b14db48ecb69299e0956cd (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]) successfully deleted
I20260618 13:59:23.476615 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 29bf263d20934ed6a6efb4b215252c00 (table default.Table_5f8fbc1f56c1484cb99b55730190bbcf [id=7923680bde144e3dbb78c922d09540b9]) successfully deleted
I20260618 13:59:23.476764 28802 ts_tablet_manager.cc:1950] T dca54111fea047d9ae0fabe376d89382 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:23.477289 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet 864f2602840c40fc8606b3784e671cba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.477730 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_f37d21a9a59249d6a0b87c3788ff4873"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:23.478013 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet dca54111fea047d9ae0fabe376d89382 (table default.Table_77ced729210e46da993ea9e57db1d716 [id=fcccef3123f14001be3f111806a209a0]) successfully deleted
W20260618 13:59:23.478349 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f37d21a9a59249d6a0b87c3788ff4873 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.478549 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet d6e3060118e844559803a649224d7474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.480046 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_768d783dc0a7416686369250c3cd855c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:23.480137 28520 tablet_service.cc:1511] Processing CreateTablet for tablet 6b8f273def3e4b7caa4d624cf87a106c (DEFAULT_TABLE table=default.Table_cb42691367014b83a1ecbc738cbe0ab9 [id=cfb4ca8c4e884ccc8602be52d0b39100]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.480535 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d23b5d2d67945a6ab9591969acb0d27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.480834 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet b8389b77e6b8407db38e07e85f7509b1 with delete_type TABLET_DATA_DELETED (replaced by tablet d9c00003300246ada2a356e3e1cf684b) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:23.481896 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_768d783dc0a7416686369250c3cd855c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.482018 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "411b0c333ae7450f9ada90afee553615" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:23.483448 28388 tablet_service.cc:1558] Processing DeleteTablet for tablet b16f7d3c7837498e996bba1818110642 with delete_type TABLET_DATA_DELETED (replaced by tablet d1baf86497d442dbbbddbc3498bebbee) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:23.483486 28484 leader_election.cc:343] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:23.483752 28810 leader_election.cc:290] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:23.484030 28810 raft_consensus.cc:493] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:23.484207 28810 raft_consensus.cc:515] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:23.484465 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 864f2602840c40fc8606b3784e671cba because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 864f2602840c40fc8606b3784e671cba already in progress: creating tablet
I20260618 13:59:23.484911 28810 leader_election.cc:290] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:23.484908 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet b16f7d3c7837498e996bba1818110642 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b16f7d3c7837498e996bba1818110642 already in progress: creating tablet
I20260618 13:59:23.486342 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b328882af549a78535cdd55c6ea09c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:23.486608 28548 raft_consensus.cc:3060] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:23.489715 28484 leader_election.cc:343] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 059716cec7f94ea6ad8989fecc6f832a
I20260618 13:59:23.498677 28800 ts_tablet_manager.cc:1929] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:23.502336 28800 log.cc:1199] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/29bf263d20934ed6a6efb4b215252c00
I20260618 13:59:23.502925 28800 ts_tablet_manager.cc:1950] T 29bf263d20934ed6a6efb4b215252c00 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:23.504601 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 29bf263d20934ed6a6efb4b215252c00 (table default.Table_5f8fbc1f56c1484cb99b55730190bbcf [id=7923680bde144e3dbb78c922d09540b9]) successfully deleted
I20260618 13:59:23.505512 28800 tablet_replica.cc:333] T 18fb8adfc8ee40e18a1da5f978317124 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:23.505872 28800 raft_consensus.cc:2243] T 18fb8adfc8ee40e18a1da5f978317124 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.508230 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "411b0c333ae7450f9ada90afee553615" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:23.508538 28410 raft_consensus.cc:2468] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:23.508961 28934 raft_consensus.cc:493] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:23.509163 28934 raft_consensus.cc:515] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.509902 28934 leader_election.cc:290] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:23.510000 28799 tablet_bootstrap.cc:492] T 8be4374ae7cf4907940ae90299bf9ba0 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:23.510324 28799 ts_tablet_manager.cc:1403] T 8be4374ae7cf4907940ae90299bf9ba0 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.154s user 0.015s sys 0.006s
I20260618 13:59:23.511117 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab50e1f17a74eee8cad942525002996" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:23.511852 28799 raft_consensus.cc:359] T 8be4374ae7cf4907940ae90299bf9ba0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.512115 28799 raft_consensus.cc:385] T 8be4374ae7cf4907940ae90299bf9ba0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.512225 28799 raft_consensus.cc:740] T 8be4374ae7cf4907940ae90299bf9ba0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
W20260618 13:59:23.512329 28346 leader_election.cc:343] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:23.512539 28799 consensus_queue.cc:260] T 8be4374ae7cf4907940ae90299bf9ba0 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.515031 28799 ts_tablet_manager.cc:1434] T 8be4374ae7cf4907940ae90299bf9ba0 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260618 13:59:23.516219 28799 tablet_bootstrap.cc:492] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:23.517330 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 8be4374ae7cf4907940ae90299bf9ba0 with delete_type TABLET_DATA_DELETED (replaced by tablet 092a9aba9c6e48f5be9cf5d55ba93678) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.517761 28802 tablet_replica.cc:333] T 8be4374ae7cf4907940ae90299bf9ba0 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:23.517989 28802 raft_consensus.cc:2243] T 8be4374ae7cf4907940ae90299bf9ba0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.518188 28802 raft_consensus.cc:2272] T 8be4374ae7cf4907940ae90299bf9ba0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:23.518154 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d7a97176f72349cda221baaea61d18ad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d7a97176f72349cda221baaea61d18ad already in progress: creating tablet
I20260618 13:59:23.518775 28800 raft_consensus.cc:2272] T 18fb8adfc8ee40e18a1da5f978317124 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.519133 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b8f273def3e4b7caa4d624cf87a106c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.519419 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet b16f7d3c7837498e996bba1818110642 with delete_type TABLET_DATA_DELETED (replaced by tablet d1baf86497d442dbbbddbc3498bebbee) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.519568 28802 ts_tablet_manager.cc:1916] T 8be4374ae7cf4907940ae90299bf9ba0 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.506083 28797 tablet_bootstrap.cc:492] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:23.520207 28797 ts_tablet_manager.cc:1403] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.179s user 0.003s sys 0.002s
I20260618 13:59:23.521713 28797 raft_consensus.cc:359] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.522010 28797 raft_consensus.cc:385] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.522166 28797 raft_consensus.cc:740] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.520784 28800 ts_tablet_manager.cc:1916] T 18fb8adfc8ee40e18a1da5f978317124 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.522666 28799 tablet_bootstrap.cc:654] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.523633 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 864f2602840c40fc8606b3784e671cba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.524252 28797 consensus_queue.cc:260] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.527149 28799 tablet_bootstrap.cc:492] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:23.527290 28799 ts_tablet_manager.cc:1403] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260618 13:59:23.528167 28799 raft_consensus.cc:359] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.528532 28799 raft_consensus.cc:385] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.528645 28799 raft_consensus.cc:740] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.528888 28799 consensus_queue.cc:260] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.530341 28548 raft_consensus.cc:2468] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:23.533387 28799 ts_tablet_manager.cc:1434] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260618 13:59:23.533984 28802 ts_tablet_manager.cc:1929] T 8be4374ae7cf4907940ae90299bf9ba0 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.534200 28802 log.cc:1199] T 8be4374ae7cf4907940ae90299bf9ba0 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/8be4374ae7cf4907940ae90299bf9ba0
I20260618 13:59:23.534593 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 5df99b28e9ae4b6bbad0fe0dafd125d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.535061 28802 ts_tablet_manager.cc:1950] T 8be4374ae7cf4907940ae90299bf9ba0 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:23.536106 28802 tablet_replica.cc:333] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:23.536307 28802 raft_consensus.cc:2243] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.536438 28802 raft_consensus.cc:2272] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.536339 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_48fb6a23c02e4c7788107c2fcdcfdd57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:23.536455 28224 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_7bdb43ababee47dc9ea2c9ec755fdbce" } new_table_name: "default.Table_f7398bc6e2f242d592730693cfb0c3b3" modify_external_catalogs: true
I20260618 13:59:23.536514 28229 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_4cd055d9d2134a539e6a1f8c4741ad60" } new_table_name: "default.Table_37ae7dae52a34dbeb8420e8aa7529336" modify_external_catalogs: true
W20260618 13:59:23.536973 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_48fb6a23c02e4c7788107c2fcdcfdd57 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.538566 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 8be4374ae7cf4907940ae90299bf9ba0 (table default.Table_0a13b90182654346acbbc8f47a56436d [id=c31d65adb55f458c9ced14d598464f89]) successfully deleted
I20260618 13:59:23.540406 28227 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_41e8e8de10ce4085b62fe5dbb7b4dcdc" } modify_external_catalogs: true
I20260618 13:59:23.540602 28227 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_41e8e8de10ce4085b62fe5dbb7b4dcdc" } modify_external_catalogs: true
I20260618 13:59:23.541816 28618 leader_election.cc:304] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:23.542273 28806 raft_consensus.cc:2804] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:23.542654 28810 raft_consensus.cc:493] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:23.542814 28810 raft_consensus.cc:515] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.544322 28810 leader_election.cc:290] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:23.541386 28483 leader_election.cc:343] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 059716cec7f94ea6ad8989fecc6f832a
I20260618 13:59:23.544732 28799 tablet_bootstrap.cc:492] T e025b64380314546af8a1ab90c817825 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:23.545469 28483 leader_election.cc:304] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:23.545965 28483 leader_election.cc:304] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2; no voters: d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:23.547367 28802 ts_tablet_manager.cc:1916] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.551451 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311ba4dd83984f379c8f4f8274bc26c1" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:23.551704 28682 raft_consensus.cc:2468] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:23.552296 28484 leader_election.cc:304] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:23.553117 28810 raft_consensus.cc:2749] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:23.554019 28483 leader_election.cc:343] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 311ba4dd83984f379c8f4f8274bc26c1
I20260618 13:59:23.554368 28802 ts_tablet_manager.cc:1929] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.554381 29057 raft_consensus.cc:2804] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:23.554510 29057 raft_consensus.cc:493] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:23.554626 28802 log.cc:1199] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/5df99b28e9ae4b6bbad0fe0dafd125d6
I20260618 13:59:23.554666 29057 raft_consensus.cc:3060] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:23.555020 28802 ts_tablet_manager.cc:1950] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
W20260618 13:59:23.555958 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 751cd9bd961b4f56a588ed02a05f0212 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 751cd9bd961b4f56a588ed02a05f0212 already in progress: creating tablet
I20260618 13:59:23.556110 28221 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_ba909970a26742b4b1006b747874b417" } new_table_name: "default.Table_4f1453c90c9f4b49b95607b12ad0bb65" modify_external_catalogs: true
I20260618 13:59:23.556126 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 5df99b28e9ae4b6bbad0fe0dafd125d6 (table default.Table_2bf9616ba425458d8c7a4e9a6854d122 [id=e3f4e13202da42a786d3376ba662b31a]) successfully deleted
W20260618 13:59:23.556257 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet fca76774f119435899188313a33dad98 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fca76774f119435899188313a33dad98 already in progress: creating tablet
W20260618 13:59:23.558562 28224 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 39b06ae7444a48dbbec9e096fd7e4052 has no consensus state
I20260618 13:59:23.553814 28810 raft_consensus.cc:2804] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:23.561980 28810 raft_consensus.cc:493] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:23.562162 28810 raft_consensus.cc:3060] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:23.565332 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 5332b6bb76ea419086341738fd135b24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:23.566933 28229 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c84189a4b1194c87bbdf81f6e0ed56e4 consensus state has no leader
I20260618 13:59:23.567405 28227 catalog_manager.cc:5978] Not sending DeleteTablet requests; no consensus state for tablet 6d31841fc5294ea9911e179d15ecdda2
W20260618 13:59:23.567786 28221 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fddea8fba4da40d7a5780e6750caf5e9 consensus state has no leader
I20260618 13:59:23.568553 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab50e1f17a74eee8cad942525002996" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:23.570392 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet 059716cec7f94ea6ad8989fecc6f832a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.571358 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_2ddc2d9b219945ae96fba0f3c312f2ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:23.571947 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ddc2d9b219945ae96fba0f3c312f2ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.573536 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_3cdafadde7c146d38bdb87c358269c35"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:23.574121 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3cdafadde7c146d38bdb87c358269c35 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.574766 28224 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_046616bed79749d5923cbd9ab9dff2b4" } modify_external_catalogs: true
I20260618 13:59:23.574939 28224 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_046616bed79749d5923cbd9ab9dff2b4" } modify_external_catalogs: true
I20260618 13:59:23.576885 28227 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_cb42691367014b83a1ecbc738cbe0ab9" } new_table_name: "default.Table_104d61375afe42e1a0450f2a190bd307" modify_external_catalogs: true
I20260618 13:59:23.579743 28229 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_b8741a7fd29f4e78a948b1c419595a85"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:23.580279 28229 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b8741a7fd29f4e78a948b1c419595a85 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.580432 28800 ts_tablet_manager.cc:1929] T 18fb8adfc8ee40e18a1da5f978317124 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.580564 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet 311ba4dd83984f379c8f4f8274bc26c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.580911 28801 tablet_replica.cc:333] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:23.582002 28798 ts_tablet_manager.cc:1434] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.162s user 0.009s sys 0.005s
I20260618 13:59:23.582060 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 311ba4dd83984f379c8f4f8274bc26c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:23.582846 28346 leader_election.cc:343] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:23.582984 28346 leader_election.cc:304] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:23.583545 28934 raft_consensus.cc:2749] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:23.584621 28797 ts_tablet_manager.cc:1434] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.064s user 0.001s sys 0.001s
I20260618 13:59:23.585536 28801 raft_consensus.cc:2243] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.585796 28801 raft_consensus.cc:2272] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.587421 28226 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_e0fda74dbcde4ee7bc2113a6c31cc9fc" } new_table_name: "default.Table_6f3c7c1d308e4901887fc9b602b38e4f" modify_external_catalogs: true
I20260618 13:59:23.592284 28797 tablet_bootstrap.cc:492] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:23.594877 28799 tablet_bootstrap.cc:654] T e025b64380314546af8a1ab90c817825 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.596191 28798 tablet_bootstrap.cc:492] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:23.596730 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 18fb8adfc8ee40e18a1da5f978317124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.596786 29055 raft_consensus.cc:493] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:23.597105 28801 ts_tablet_manager.cc:1916] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.580606 28800 log.cc:1199] T 18fb8adfc8ee40e18a1da5f978317124 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/18fb8adfc8ee40e18a1da5f978317124
I20260618 13:59:23.599231 28806 raft_consensus.cc:697] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:23.599792 28806 consensus_queue.cc:237] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.600975 28810 raft_consensus.cc:515] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.601495 28797 tablet_bootstrap.cc:654] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.601691 28810 leader_election.cc:290] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:23.602185 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "411b0c333ae7450f9ada90afee553615" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:23.603040 29057 raft_consensus.cc:515] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.603698 29057 leader_election.cc:290] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:23.606259 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311ba4dd83984f379c8f4f8274bc26c1" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:23.606453 28682 raft_consensus.cc:3060] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:23.608287 29055 raft_consensus.cc:515] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.609337 29055 leader_election.cc:290] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:23.609639 28682 raft_consensus.cc:2468] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:23.610041 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "411b0c333ae7450f9ada90afee553615" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:23.610224 28410 raft_consensus.cc:3060] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:23.611404 28798 tablet_bootstrap.cc:654] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:23.613549 28483 leader_election.cc:343] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 311ba4dd83984f379c8f4f8274bc26c1
W20260618 13:59:23.613794 28483 leader_election.cc:343] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 18fb8adfc8ee40e18a1da5f978317124
I20260618 13:59:23.617455 28410 raft_consensus.cc:2468] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:23.617966 28483 leader_election.cc:304] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2; no voters:
I20260618 13:59:23.618274 29055 raft_consensus.cc:2804] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:23.618464 29055 raft_consensus.cc:697] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:23.618913 29055 consensus_queue.cc:237] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.621363 28801 ts_tablet_manager.cc:1929] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:23.621526 28801 log.cc:1199] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/29bf263d20934ed6a6efb4b215252c00
I20260618 13:59:23.622038 28801 ts_tablet_manager.cc:1950] T 29bf263d20934ed6a6efb4b215252c00 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:23.622818 28801 tablet_replica.cc:333] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:23.623059 28801 raft_consensus.cc:2243] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.623284 28801 raft_consensus.cc:2272] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.623940 28801 ts_tablet_manager.cc:1916] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:23.627671 28484 leader_election.cc:343] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:23.628338 28484 leader_election.cc:304] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:23.628832 29057 raft_consensus.cc:2804] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:23.629051 29057 raft_consensus.cc:697] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:23.629415 29057 consensus_queue.cc:237] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:23.639966 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 already in progress: creating tablet
I20260618 13:59:23.640107 28225 master_service.cc:960] ReplaceTablet: received request to replace tablet c84189a4b1194c87bbdf81f6e0ed56e4 from {username='slave'} at 127.0.0.1:45138
W20260618 13:59:23.640296 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 092a9aba9c6e48f5be9cf5d55ba93678 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 092a9aba9c6e48f5be9cf5d55ba93678 already in progress: creating tablet
I20260618 13:59:23.639930 28223 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_af2d1c7150e84f96807cda9b1dabd299" } modify_external_catalogs: true
W20260618 13:59:23.640522 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 283523e5b22843ba935eabe6283d3a99 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 283523e5b22843ba935eabe6283d3a99 already in progress: creating tablet
I20260618 13:59:23.640012 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_ea7b830f69254dc49cbc9e3c17c3be85"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:23.640558 28223 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_af2d1c7150e84f96807cda9b1dabd299" } modify_external_catalogs: true
W20260618 13:59:23.641001 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ea7b830f69254dc49cbc9e3c17c3be85 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.642161 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18fb8adfc8ee40e18a1da5f978317124" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:23.642834 28484 leader_election.cc:343] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: Tablet not RUNNING: STOPPED
I20260618 13:59:23.642982 28484 leader_election.cc:304] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:23.643244 28810 raft_consensus.cc:2749] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:23.651245 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 29bf263d20934ed6a6efb4b215252c00 (table default.Table_5f8fbc1f56c1484cb99b55730190bbcf [id=7923680bde144e3dbb78c922d09540b9]) successfully deleted
W20260618 13:59:23.651539 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 4ebed81a21b14db48ecb69299e0956cd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ebed81a21b14db48ecb69299e0956cd already in progress: creating tablet
I20260618 13:59:23.652159 28220 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_74c0d9b4f3724e278144c5fc30d8c2d7" } modify_external_catalogs: true
I20260618 13:59:23.652333 28220 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_74c0d9b4f3724e278144c5fc30d8c2d7" } modify_external_catalogs: true
I20260618 13:59:23.653064 28798 tablet_bootstrap.cc:492] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:23.653851 28798 ts_tablet_manager.cc:1403] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.072s user 0.002s sys 0.009s
I20260618 13:59:23.653926 28800 ts_tablet_manager.cc:1950] T 18fb8adfc8ee40e18a1da5f978317124 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:23.655200 28798 raft_consensus.cc:359] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.655277 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 18fb8adfc8ee40e18a1da5f978317124 (table default.Table_bf7cbaec816a4c04a225ff31f57123ed [id=f34f77924d464341afa518830ba7bbbe]) successfully deleted
I20260618 13:59:23.655624 28800 tablet_replica.cc:333] T 5332b6bb76ea419086341738fd135b24 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:23.655879 28800 raft_consensus.cc:2243] T 5332b6bb76ea419086341738fd135b24 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.656095 28800 raft_consensus.cc:2272] T 5332b6bb76ea419086341738fd135b24 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.656253 28798 raft_consensus.cc:385] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.657579 28798 raft_consensus.cc:740] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.659773 28800 ts_tablet_manager.cc:1916] T 5332b6bb76ea419086341738fd135b24 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.664907 28799 tablet_bootstrap.cc:492] T e025b64380314546af8a1ab90c817825 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:23.667418 28797 tablet_bootstrap.cc:492] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:23.667523 28229 master_service.cc:960] ReplaceTablet: received request to replace tablet e025b64380314546af8a1ab90c817825 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:23.667580 28797 ts_tablet_manager.cc:1403] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.075s user 0.001s sys 0.002s
I20260618 13:59:23.668444 28797 raft_consensus.cc:359] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.668648 28797 raft_consensus.cc:385] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.668725 28797 raft_consensus.cc:740] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.665089 28799 ts_tablet_manager.cc:1403] T e025b64380314546af8a1ab90c817825 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.120s user 0.002s sys 0.002s
I20260618 13:59:23.670656 28799 raft_consensus.cc:359] T e025b64380314546af8a1ab90c817825 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.670845 28799 raft_consensus.cc:385] T e025b64380314546af8a1ab90c817825 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:23.670917 28799 raft_consensus.cc:740] T e025b64380314546af8a1ab90c817825 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:23.671170 28799 consensus_queue.cc:260] T e025b64380314546af8a1ab90c817825 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.672892 28799 ts_tablet_manager.cc:1434] T e025b64380314546af8a1ab90c817825 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:23.673529 28797 consensus_queue.cc:260] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.674696 28799 tablet_bootstrap.cc:492] T 68b8293a7c924a449307ab88a1cf9290 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:23.675046 28227 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6b8f273def3e4b7caa4d624cf87a106c consensus state has no leader
W20260618 13:59:23.675397 28226 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 02b5cc08ca0c4e719e0414b345211ba5 consensus state has no leader
I20260618 13:59:23.676466 28800 ts_tablet_manager.cc:1929] T 5332b6bb76ea419086341738fd135b24 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.676592 28800 log.cc:1199] T 5332b6bb76ea419086341738fd135b24 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/5332b6bb76ea419086341738fd135b24
I20260618 13:59:23.676921 28226 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 411b0c333ae7450f9ada90afee553615 (table default.Table_5d0d3ddc1f4a4badbb853268cb2bba65 [id=cd89063e7d294d3fbb3b58358e781c90]). Expected version 2 got 0
I20260618 13:59:23.677008 28800 ts_tablet_manager.cc:1950] T 5332b6bb76ea419086341738fd135b24 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:23.678742 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 39b06ae7444a48dbbec9e096fd7e4052 (DEFAULT_TABLE table=default.Table_f7398bc6e2f242d592730693cfb0c3b3 [id=1b34fcbd96004d8c9084c7360106aa15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.679215 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39b06ae7444a48dbbec9e096fd7e4052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.679538 28797 ts_tablet_manager.cc:1434] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260618 13:59:23.681733 28797 tablet_bootstrap.cc:492] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:23.681967 28800 tablet_replica.cc:333] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:23.682148 28800 raft_consensus.cc:2243] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:23.683579 28797 tablet_bootstrap.cc:654] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.683832 28226 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 411b0c333ae7450f9ada90afee553615 (table default.Table_5d0d3ddc1f4a4badbb853268cb2bba65 [id=cd89063e7d294d3fbb3b58358e781c90]). Expected version 2 got 0
I20260618 13:59:23.684249 28385 tablet_service.cc:1511] Processing CreateTablet for tablet f278d0acbb0c4a0ca61640bc142e6335 (DEFAULT_TABLE table=default.Table_5b3dc6cb553c45638fde791608253e3c [id=0d83fc68255c458587390c58b12e00d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.686074 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 15180aefba0544329daee74308ef5010 (DEFAULT_TABLE table=default.Table_b9f02faebbe34f9299f751f0e9f17d57 [id=42a2a24590834abc9537669aaf223a7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.686672 28390 tablet_service.cc:1511] Processing CreateTablet for tablet e6de1293572f4e75af3417f281b9a142 (DEFAULT_TABLE table=default.Table_f75651a1e55f4271b39388713dcd74aa [id=6705bb69561d435496d424c0d64f9879]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.687474 28387 tablet_service.cc:1558] Processing DeleteTablet for tablet e025b64380314546af8a1ab90c817825 with delete_type TABLET_DATA_DELETED (replaced by tablet ece7909d056c4ba8a5dca6423f0c1781) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.687917 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f278d0acbb0c4a0ca61640bc142e6335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.688402 28382 tablet_service.cc:1511] Processing CreateTablet for tablet 75824a5d2f7e47838b60e6a2b785c6b2 (DEFAULT_TABLE table=default.Table_d838306823ff4e22a68efe0a8cb751ff [id=bbfd0a6a47db4b1b8c534405b1068d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.688813 28382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75824a5d2f7e47838b60e6a2b785c6b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.687733 28388 tablet_service.cc:1511] Processing CreateTablet for tablet 02b5cc08ca0c4e719e0414b345211ba5 (DEFAULT_TABLE table=default.Table_e0fda74dbcde4ee7bc2113a6c31cc9fc [id=7aa3803c01134af6b073952eb9843282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.692896 28802 tablet_replica.cc:333] T e025b64380314546af8a1ab90c817825 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:23.693146 28802 raft_consensus.cc:2243] T e025b64380314546af8a1ab90c817825 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.693483 28802 raft_consensus.cc:2272] T e025b64380314546af8a1ab90c817825 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.693843 28390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6de1293572f4e75af3417f281b9a142. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.695556 28656 tablet_service.cc:1511] Processing CreateTablet for tablet 15180aefba0544329daee74308ef5010 (DEFAULT_TABLE table=default.Table_b9f02faebbe34f9299f751f0e9f17d57 [id=42a2a24590834abc9537669aaf223a7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.695787 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02b5cc08ca0c4e719e0414b345211ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.697435 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15180aefba0544329daee74308ef5010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.698823 28387 tablet_service.cc:1511] Processing CreateTablet for tablet aef7c1f111634c98af046e73fa9ffdeb (DEFAULT_TABLE table=default.Table_191169025bff42cab93cf42503fa7220 [id=9c29cca38acc493aaec62cf3b0d93fcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.700011 28802 ts_tablet_manager.cc:1916] T e025b64380314546af8a1ab90c817825 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.700410 28387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aef7c1f111634c98af046e73fa9ffdeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.700583 28649 tablet_service.cc:1511] Processing CreateTablet for tablet f278d0acbb0c4a0ca61640bc142e6335 (DEFAULT_TABLE table=default.Table_5b3dc6cb553c45638fde791608253e3c [id=0d83fc68255c458587390c58b12e00d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.701674 28648 tablet_service.cc:1511] Processing CreateTablet for tablet 75824a5d2f7e47838b60e6a2b785c6b2 (DEFAULT_TABLE table=default.Table_d838306823ff4e22a68efe0a8cb751ff [id=bbfd0a6a47db4b1b8c534405b1068d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.702383 28386 tablet_service.cc:1511] Processing CreateTablet for tablet 8fd9fa9675234db2abff694a517c0ea8 (DEFAULT_TABLE table=default.Table_726d3825b282496fb87c67a8975e3fcc [id=f914bcce8551443caa218e7a92967307]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.702875 28858 rpc_server.cc:307] RPC server started. Bound to: 127.27.100.61:45967
I20260618 13:59:23.703284 28384 tablet_service.cc:1511] Processing CreateTablet for tablet 72fd63e8245d4042b091593881ba11b6 (DEFAULT_TABLE table=default.Table_b0e3e2341f764392af42b949d03dd1f6 [id=27b6a19a28f24f2b8a758bbff06312d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.703620 28660 tablet_service.cc:1511] Processing CreateTablet for tablet e6de1293572f4e75af3417f281b9a142 (DEFAULT_TABLE table=default.Table_f75651a1e55f4271b39388713dcd74aa [id=6705bb69561d435496d424c0d64f9879]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.704975 28934 raft_consensus.cc:493] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:23.706183 28653 tablet_service.cc:1511] Processing CreateTablet for tablet aef7c1f111634c98af046e73fa9ffdeb (DEFAULT_TABLE table=default.Table_191169025bff42cab93cf42503fa7220 [id=9c29cca38acc493aaec62cf3b0d93fcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.706341 28650 tablet_service.cc:1511] Processing CreateTablet for tablet 39b06ae7444a48dbbec9e096fd7e4052 (DEFAULT_TABLE table=default.Table_f7398bc6e2f242d592730693cfb0c3b3 [id=1b34fcbd96004d8c9084c7360106aa15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.706779 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15180aefba0544329daee74308ef5010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.707093 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aef7c1f111634c98af046e73fa9ffdeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.707247 28659 tablet_service.cc:1511] Processing CreateTablet for tablet 02b5cc08ca0c4e719e0414b345211ba5 (DEFAULT_TABLE table=default.Table_e0fda74dbcde4ee7bc2113a6c31cc9fc [id=7aa3803c01134af6b073952eb9843282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.707355 28649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f278d0acbb0c4a0ca61640bc142e6335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.707561 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02b5cc08ca0c4e719e0414b345211ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.707921 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fd9fa9675234db2abff694a517c0ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.708069 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6de1293572f4e75af3417f281b9a142. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.708304 28650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39b06ae7444a48dbbec9e096fd7e4052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.708606 28648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75824a5d2f7e47838b60e6a2b785c6b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.718876 28858 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-1/data/info.pb
I20260618 13:59:23.719081 29085 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.100.61:45967 every 8 connection(s)
I20260618 13:59:23.722949 28228 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_9e1e848ca5c94952ab3928c5feafe2f1" } modify_external_catalogs: true
I20260618 13:59:23.723134 28228 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_9e1e848ca5c94952ab3928c5feafe2f1" } modify_external_catalogs: true
I20260618 13:59:23.725018 28934 raft_consensus.cc:515] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.725268 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_6ba7e86569384c888e4208d3414512ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:23.725373 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_663ed02889aa45938d7d1316d4d19c06"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:23.725687 28934 leader_election.cc:290] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:23.725764 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6ba7e86569384c888e4208d3414512ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:23.725878 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_663ed02889aa45938d7d1316d4d19c06 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.726881 28391 tablet_service.cc:1511] Processing CreateTablet for tablet cd06bb7cffc34412b420d70435fce373 (DEFAULT_TABLE table=default.Table_74fc7c21dc664ff8bfe0d25aee399bdb [id=d224150fd3a340fc96e75c0b96146b11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.727744 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72fd63e8245d4042b091593881ba11b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.727963 28048 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu as pid 28858
I20260618 13:59:23.730782 28381 tablet_service.cc:1511] Processing CreateTablet for tablet b51d0851efb34eaa9e9e4be6b292ef62 (DEFAULT_TABLE table=default.Table_7d317fa28e3142a19d15c8360151c357 [id=bba3ab8821ca48a5aa1104094d78892a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.732805 28380 tablet_service.cc:1511] Processing CreateTablet for tablet 6eea4d59935d40aa97dc10ec7469aa56 (DEFAULT_TABLE table=default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.733078 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd06bb7cffc34412b420d70435fce373. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.735458 28381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b51d0851efb34eaa9e9e4be6b292ef62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.736474 28380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eea4d59935d40aa97dc10ec7469aa56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.744166 28227 catalog_manager.cc:5696] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97859748af2c41879ba2a668dda999d2 (127.27.100.2). New cstate: current_term: 1 leader_uuid: "97859748af2c41879ba2a668dda999d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:23.744632 28227 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 411b0c333ae7450f9ada90afee553615 (table default.Table_5d0d3ddc1f4a4badbb853268cb2bba65 [id=cd89063e7d294d3fbb3b58358e781c90]). Expected version 2 got 0
I20260618 13:59:23.746061 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_176a739c87ee45798e2470f8ec00d632"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:23.746675 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_176a739c87ee45798e2470f8ec00d632 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.749092 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 5332b6bb76ea419086341738fd135b24 (table default.Table_e4cb8c97170c465980e9eeb3f6e4f88c [id=116d7007e8754aab8d37956bcb596f13]) successfully deleted
I20260618 13:59:23.664440 28801 ts_tablet_manager.cc:1929] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.770021 28801 log.cc:1199] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/31b328882af549a78535cdd55c6ea09c
I20260618 13:59:23.770133 28798 consensus_queue.cc:260] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.770521 28801 ts_tablet_manager.cc:1950] T 31b328882af549a78535cdd55c6ea09c P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:23.770798 28798 ts_tablet_manager.cc:1434] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.117s user 0.002s sys 0.000s
W20260618 13:59:23.771219 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet b16f7d3c7837498e996bba1818110642 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b16f7d3c7837498e996bba1818110642 already in progress: creating tablet
I20260618 13:59:23.694837 28525 tablet_service.cc:1511] Processing CreateTablet for tablet 39b06ae7444a48dbbec9e096fd7e4052 (DEFAULT_TABLE table=default.Table_f7398bc6e2f242d592730693cfb0c3b3 [id=1b34fcbd96004d8c9084c7360106aa15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.772022 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39b06ae7444a48dbbec9e096fd7e4052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.772320 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 31b328882af549a78535cdd55c6ea09c (table default.Table_823368bbdf8242739a6022f65280f5a2 [id=28f3530b169a44eb9c262a718c8ea6d3]) successfully deleted
I20260618 13:59:23.695785 28524 tablet_service.cc:1511] Processing CreateTablet for tablet 15180aefba0544329daee74308ef5010 (DEFAULT_TABLE table=default.Table_b9f02faebbe34f9299f751f0e9f17d57 [id=42a2a24590834abc9537669aaf223a7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.749143 28229 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_259b4dd1855b42a8824f00306cb5223a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:23.774691 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15180aefba0544329daee74308ef5010. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:23.774768 28229 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_259b4dd1855b42a8824f00306cb5223a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.697708 28520 tablet_service.cc:1511] Processing CreateTablet for tablet 02b5cc08ca0c4e719e0414b345211ba5 (DEFAULT_TABLE table=default.Table_e0fda74dbcde4ee7bc2113a6c31cc9fc [id=7aa3803c01134af6b073952eb9843282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.700965 28523 tablet_service.cc:1511] Processing CreateTablet for tablet f278d0acbb0c4a0ca61640bc142e6335 (DEFAULT_TABLE table=default.Table_5b3dc6cb553c45638fde791608253e3c [id=0d83fc68255c458587390c58b12e00d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.777405 28220 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 8b59255b14df4158afe37cb8890e7415
I20260618 13:59:23.778096 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_0e9f574416d84a0f826fa9a1fcc56210"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:23.778631 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0e9f574416d84a0f826fa9a1fcc56210 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.780021 28225 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet c84189a4b1194c87bbdf81f6e0ed56e4
I20260618 13:59:23.780427 28225 catalog_manager.cc:6784] ReplaceTablet: tablet c84189a4b1194c87bbdf81f6e0ed56e4 deleted and replaced by tablet be00bfa9bc1a4b83b18bfad05ba1495e
I20260618 13:59:23.780663 28225 master_service.cc:960] ReplaceTablet: received request to replace tablet 0a51e830e3014726b1d1c2608973f036 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:23.781116 28225 master_service.cc:960] ReplaceTablet: received request to replace tablet 4a01dfb4d59549399c94640f16045555 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:23.781607 28225 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_061bec38727b4ee69096fe3b50296f94" } new_table_name: "default.Table_dac9c04f74664cce9c065edc3e491bf6" modify_external_catalogs: true
I20260618 13:59:23.782233 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0c9d4131ea24809ac081e7a73a21e1c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:23.782657 28223 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet e1a75029992645c5b942c0ec07862bb4
I20260618 13:59:23.782953 28657 tablet_service.cc:1511] Processing CreateTablet for tablet 6eea4d59935d40aa97dc10ec7469aa56 (DEFAULT_TABLE table=default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.783200 28223 master_service.cc:960] ReplaceTablet: received request to replace tablet 4c2b924a096b479cad31c29bf7c99172 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:23.783335 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eea4d59935d40aa97dc10ec7469aa56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.702111 28522 tablet_service.cc:1511] Processing CreateTablet for tablet 75824a5d2f7e47838b60e6a2b785c6b2 (DEFAULT_TABLE table=default.Table_d838306823ff4e22a68efe0a8cb751ff [id=bbfd0a6a47db4b1b8c534405b1068d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.704833 28519 tablet_service.cc:1511] Processing CreateTablet for tablet e6de1293572f4e75af3417f281b9a142 (DEFAULT_TABLE table=default.Table_f75651a1e55f4271b39388713dcd74aa [id=6705bb69561d435496d424c0d64f9879]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.719579 28529 tablet_service.cc:1511] Processing CreateTablet for tablet aef7c1f111634c98af046e73fa9ffdeb (DEFAULT_TABLE table=default.Table_191169025bff42cab93cf42503fa7220 [id=9c29cca38acc493aaec62cf3b0d93fcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.784065 28648 tablet_service.cc:1511] Processing CreateTablet for tablet 72fd63e8245d4042b091593881ba11b6 (DEFAULT_TABLE table=default.Table_b0e3e2341f764392af42b949d03dd1f6 [id=27b6a19a28f24f2b8a758bbff06312d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.720219 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 72fd63e8245d4042b091593881ba11b6 (DEFAULT_TABLE table=default.Table_b0e3e2341f764392af42b949d03dd1f6 [id=27b6a19a28f24f2b8a758bbff06312d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.784482 29086 sys_catalog.cc:263] Verifying existing consensus state
W20260618 13:59:23.784808 28346 leader_election.cc:343] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:23.785735 28648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72fd63e8245d4042b091593881ba11b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.787164 28651 tablet_service.cc:1511] Processing CreateTablet for tablet b51d0851efb34eaa9e9e4be6b292ef62 (DEFAULT_TABLE table=default.Table_7d317fa28e3142a19d15c8360151c357 [id=bba3ab8821ca48a5aa1104094d78892a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.787493 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b51d0851efb34eaa9e9e4be6b292ef62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.789579 28224 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 3752f49c9fc64071a45e0ce4ff4cfef2
I20260618 13:59:23.789911 29086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Bootstrap starting.
I20260618 13:59:23.792513 28663 tablet_service.cc:1511] Processing CreateTablet for tablet cd06bb7cffc34412b420d70435fce373 (DEFAULT_TABLE table=default.Table_74fc7c21dc664ff8bfe0d25aee399bdb [id=d224150fd3a340fc96e75c0b96146b11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.792689 28662 tablet_service.cc:1511] Processing CreateTablet for tablet c7f42997416841a7b2c1ad347d2e9282 (DEFAULT_TABLE table=default.Table_0a13b90182654346acbbc8f47a56436d [id=c31d65adb55f458c9ced14d598464f89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.792877 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd06bb7cffc34412b420d70435fce373. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.793951 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f42997416841a7b2c1ad347d2e9282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.794373 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02b5cc08ca0c4e719e0414b345211ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.794466 28652 tablet_service.cc:1511] Processing CreateTablet for tablet ba3d38d32c7c4721bad35f0da7a9ebe5 (DEFAULT_TABLE table=default.Table_40fdb9a7e5ad4ff2bf879ec4e02a4b8d [id=c1e1069f4a1640e2a3f86b302f1dcc7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.734023 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0c9d4131ea24809ac081e7a73a21e1c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:23.795320 28652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba3d38d32c7c4721bad35f0da7a9ebe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.796394 28661 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b59255b14df4158afe37cb8890e7415 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.796886 28228 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 0fae928833e44d2ba860d6184d4c0142
I20260618 13:59:23.796896 28655 tablet_service.cc:1511] Processing CreateTablet for tablet 91d014b46a0f448b929cbc9287ead0a7 (DEFAULT_TABLE table=default.Table_62bf5ce52fb14bd089678663dd009a91 [id=713af0f85a434ed0bfb791c48f75dc9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.797217 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d014b46a0f448b929cbc9287ead0a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.799546 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b59255b14df4158afe37cb8890e7415 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.799810 28388 tablet_service.cc:1558] Processing DeleteTablet for tablet c84189a4b1194c87bbdf81f6e0ed56e4 with delete_type TABLET_DATA_DELETED (replaced by tablet be00bfa9bc1a4b83b18bfad05ba1495e) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.800282 28391 tablet_service.cc:1511] Processing CreateTablet for tablet cea4bc13eabc4dd98b4f96723498014c (DEFAULT_TABLE table=default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.800676 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cea4bc13eabc4dd98b4f96723498014c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.802676 28388 tablet_service.cc:1558] Processing DeleteTablet for tablet 3752f49c9fc64071a45e0ce4ff4cfef2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.802939 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet e1a75029992645c5b942c0ec07862bb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.808414 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aef7c1f111634c98af046e73fa9ffdeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.808730 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72fd63e8245d4042b091593881ba11b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.808832 28798 tablet_bootstrap.cc:492] T ebf502f25e9b4342bfaa58f42db7f2f5 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:23.808852 28654 tablet_service.cc:1558] Processing DeleteTablet for tablet e1a75029992645c5b942c0ec07862bb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.809070 28661 tablet_service.cc:1558] Processing DeleteTablet for tablet c84189a4b1194c87bbdf81f6e0ed56e4 with delete_type TABLET_DATA_DELETED (replaced by tablet be00bfa9bc1a4b83b18bfad05ba1495e) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.810539 28801 tablet_replica.cc:333] T d6e3060118e844559803a649224d7474 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:23.811836 28519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6de1293572f4e75af3417f281b9a142. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.812233 29086 log.cc:826] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Log is configured to *not* fsync() on all Append() calls
I20260618 13:59:23.813161 28661 tablet_service.cc:1511] Processing CreateTablet for tablet cea4bc13eabc4dd98b4f96723498014c (DEFAULT_TABLE table=default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.813652 28661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cea4bc13eabc4dd98b4f96723498014c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.814850 28654 tablet_service.cc:1558] Processing DeleteTablet for tablet 3752f49c9fc64071a45e0ce4ff4cfef2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.815900 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75824a5d2f7e47838b60e6a2b785c6b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.822513 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f278d0acbb0c4a0ca61640bc142e6335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.825342 28802 ts_tablet_manager.cc:1929] T e025b64380314546af8a1ab90c817825 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.825564 28528 tablet_service.cc:1511] Processing CreateTablet for tablet 8fd9fa9675234db2abff694a517c0ea8 (DEFAULT_TABLE table=default.Table_726d3825b282496fb87c67a8975e3fcc [id=f914bcce8551443caa218e7a92967307]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.826002 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fd9fa9675234db2abff694a517c0ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.827905 28801 raft_consensus.cc:2243] T d6e3060118e844559803a649224d7474 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.827997 28802 log.cc:1199] T e025b64380314546af8a1ab90c817825 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/e025b64380314546af8a1ab90c817825
I20260618 13:59:23.828389 28802 ts_tablet_manager.cc:1950] T e025b64380314546af8a1ab90c817825 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:23.829236 29086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260618 13:59:23.829753 28801 raft_consensus.cc:2272] T d6e3060118e844559803a649224d7474 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.829873 29086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Bootstrap complete.
I20260618 13:59:23.830458 28801 ts_tablet_manager.cc:1916] T d6e3060118e844559803a649224d7474 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.830461 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet e025b64380314546af8a1ab90c817825 (table default.Table_c940d6e3d5d34adfaf39c7b5b60044e1 [id=803d5220ce1a4dcfaf27d93c574be2ac]) successfully deleted
I20260618 13:59:23.831266 28527 tablet_service.cc:1511] Processing CreateTablet for tablet cd06bb7cffc34412b420d70435fce373 (DEFAULT_TABLE table=default.Table_74fc7c21dc664ff8bfe0d25aee399bdb [id=d224150fd3a340fc96e75c0b96146b11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.831959 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 91d014b46a0f448b929cbc9287ead0a7 (DEFAULT_TABLE table=default.Table_62bf5ce52fb14bd089678663dd009a91 [id=713af0f85a434ed0bfb791c48f75dc9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.832352 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d014b46a0f448b929cbc9287ead0a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.835882 28220 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_cc20b3f971bb46969baf1fc71e217f2c" } modify_external_catalogs: true
I20260618 13:59:23.836143 28220 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_cc20b3f971bb46969baf1fc71e217f2c" } modify_external_catalogs: true
I20260618 13:59:23.837350 28221 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_c7d24bbb3b724c98ba3da942a84843e1" } new_table_name: "default.Table_6598b23b0b264b08b63faa03d23e87a9" modify_external_catalogs: true
I20260618 13:59:23.837598 28385 tablet_service.cc:1511] Processing CreateTablet for tablet ba3d38d32c7c4721bad35f0da7a9ebe5 (DEFAULT_TABLE table=default.Table_40fdb9a7e5ad4ff2bf879ec4e02a4b8d [id=c1e1069f4a1640e2a3f86b302f1dcc7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.839367 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba3d38d32c7c4721bad35f0da7a9ebe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.839450 29086 raft_consensus.cc:359] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:23.842780 28521 tablet_service.cc:1511] Processing CreateTablet for tablet b51d0851efb34eaa9e9e4be6b292ef62 (DEFAULT_TABLE table=default.Table_7d317fa28e3142a19d15c8360151c357 [id=bba3ab8821ca48a5aa1104094d78892a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.843649 28518 tablet_service.cc:1511] Processing CreateTablet for tablet 6eea4d59935d40aa97dc10ec7469aa56 (DEFAULT_TABLE table=default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.844129 28517 tablet_service.cc:1511] Processing CreateTablet for tablet 91d014b46a0f448b929cbc9287ead0a7 (DEFAULT_TABLE table=default.Table_62bf5ce52fb14bd089678663dd009a91 [id=713af0f85a434ed0bfb791c48f75dc9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.844269 28516 tablet_service.cc:1511] Processing CreateTablet for tablet ba3d38d32c7c4721bad35f0da7a9ebe5 (DEFAULT_TABLE table=default.Table_40fdb9a7e5ad4ff2bf879ec4e02a4b8d [id=c1e1069f4a1640e2a3f86b302f1dcc7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.844357 28382 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fae928833e44d2ba860d6184d4c0142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:23.844677 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b59255b14df4158afe37cb8890e7415 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.844725 28515 tablet_service.cc:1511] Processing CreateTablet for tablet c7f42997416841a7b2c1ad347d2e9282 (DEFAULT_TABLE table=default.Table_0a13b90182654346acbbc8f47a56436d [id=c31d65adb55f458c9ced14d598464f89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.844784 28387 tablet_service.cc:1511] Processing CreateTablet for tablet c7f42997416841a7b2c1ad347d2e9282 (DEFAULT_TABLE table=default.Table_0a13b90182654346acbbc8f47a56436d [id=c31d65adb55f458c9ced14d598464f89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.845021 28512 tablet_service.cc:1558] Processing DeleteTablet for tablet e1a75029992645c5b942c0ec07862bb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.845073 28513 tablet_service.cc:1558] Processing DeleteTablet for tablet c84189a4b1194c87bbdf81f6e0ed56e4 with delete_type TABLET_DATA_DELETED (replaced by tablet be00bfa9bc1a4b83b18bfad05ba1495e) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.845156 28387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f42997416841a7b2c1ad347d2e9282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.845594 28511 tablet_service.cc:1511] Processing CreateTablet for tablet cea4bc13eabc4dd98b4f96723498014c (DEFAULT_TABLE table=default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.846793 28798 tablet_bootstrap.cc:654] T ebf502f25e9b4342bfaa58f42db7f2f5 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.848865 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd06bb7cffc34412b420d70435fce373. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.855036 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_ddb8f9243a5d47d1be63b3a371ca17ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:23.855162 28229 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_104d61375afe42e1a0450f2a190bd307" } new_table_name: "default.Table_90dfb0d3edbf419bb198321df1f067dc" modify_external_catalogs: true
W20260618 13:59:23.855682 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ddb8f9243a5d47d1be63b3a371ca17ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:23.858774 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 8b59255b14df4158afe37cb8890e7415 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b59255b14df4158afe37cb8890e7415 already in progress: creating tablet
W20260618 13:59:23.859754 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet c84189a4b1194c87bbdf81f6e0ed56e4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c84189a4b1194c87bbdf81f6e0ed56e4 already in progress: creating tablet
I20260618 13:59:23.856356 28799 tablet_bootstrap.cc:654] T 68b8293a7c924a449307ab88a1cf9290 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:23.861347 28802 tablet_replica.cc:333] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:23.861733 28802 raft_consensus.cc:2243] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.862043 28802 raft_consensus.cc:2272] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.858729 28934 raft_consensus.cc:493] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:23.862461 28934 raft_consensus.cc:515] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.862675 28521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b51d0851efb34eaa9e9e4be6b292ef62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.863165 28934 leader_election.cc:290] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:23.865077 28517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d014b46a0f448b929cbc9287ead0a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.865933 28510 tablet_service.cc:1558] Processing DeleteTablet for tablet 3752f49c9fc64071a45e0ce4ff4cfef2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.866017 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fae928833e44d2ba860d6184d4c0142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.866257 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ebed81a21b14db48ecb69299e0956cd with delete_type TABLET_DATA_DELETED (replaced by tablet 6eea4d59935d40aa97dc10ec7469aa56) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:23.866561 28346 leader_election.cc:343] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:23.866852 28346 leader_election.cc:304] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:23.867259 28934 raft_consensus.cc:2749] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:23.867106 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08a2dee3ac5a4bec96445a6001717eda" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:23.867954 28511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cea4bc13eabc4dd98b4f96723498014c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.869042 28515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f42997416841a7b2c1ad347d2e9282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.870034 28516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba3d38d32c7c4721bad35f0da7a9ebe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.870651 28220 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet bc0adedecee4405e86b0a37b9e62e352
I20260618 13:59:23.870872 28518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eea4d59935d40aa97dc10ec7469aa56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.871629 28223 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 4c2b924a096b479cad31c29bf7c99172
I20260618 13:59:23.872084 28223 catalog_manager.cc:6784] ReplaceTablet: tablet 4c2b924a096b479cad31c29bf7c99172 deleted and replaced by tablet ab521028ac1a4d0da013112d76bb169f
W20260618 13:59:23.872485 28225 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d1baf86497d442dbbbddbc3498bebbee has no consensus state
I20260618 13:59:23.872713 28382 tablet_service.cc:1558] Processing DeleteTablet for tablet bc0adedecee4405e86b0a37b9e62e352 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:23.872861 28229 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6b8f273def3e4b7caa4d624cf87a106c consensus state has no leader
I20260618 13:59:23.873431 28382 tablet_service.cc:1558] Processing DeleteTablet for tablet 4c2b924a096b479cad31c29bf7c99172 with delete_type TABLET_DATA_DELETED (replaced by tablet ab521028ac1a4d0da013112d76bb169f) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:23.874289 28221 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9c17e76ae9b44207998a43e3576908c4 consensus state has no leader
I20260618 13:59:23.875396 28802 ts_tablet_manager.cc:1916] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.892006 28799 tablet_bootstrap.cc:492] T 68b8293a7c924a449307ab88a1cf9290 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:23.892361 28799 ts_tablet_manager.cc:1403] T 68b8293a7c924a449307ab88a1cf9290 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.218s user 0.003s sys 0.001s
W20260618 13:59:23.892831 28346 leader_election.cc:343] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:23.896998 28654 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fae928833e44d2ba860d6184d4c0142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.897890 28800 pending_rounds.cc:70] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257: Trying to abort 1 pending ops.
I20260618 13:59:23.898571 28658 tablet_service.cc:1558] Processing DeleteTablet for tablet bc0adedecee4405e86b0a37b9e62e352 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.904107 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08a2dee3ac5a4bec96445a6001717eda" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:23.906049 28346 leader_election.cc:343] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:23.906498 28346 leader_election.cc:304] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:23.907402 28658 tablet_service.cc:1558] Processing DeleteTablet for tablet 4c2b924a096b479cad31c29bf7c99172 with delete_type TABLET_DATA_DELETED (replaced by tablet ab521028ac1a4d0da013112d76bb169f) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:23.911793 28934 raft_consensus.cc:2749] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:23.912434 28525 tablet_service.cc:1558] Processing DeleteTablet for tablet bc0adedecee4405e86b0a37b9e62e352 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.912636 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet 4c2b924a096b479cad31c29bf7c99172 with delete_type TABLET_DATA_DELETED (replaced by tablet ab521028ac1a4d0da013112d76bb169f) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:23.917531 28800 pending_rounds.cc:77] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298216606107885568 op_type: NO_OP noop_request { }
I20260618 13:59:23.932261 28800 raft_consensus.cc:2889] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:23.946403 28382 tablet_service.cc:1511] Processing CreateTablet for tablet 5c75278db2234b11bf95f1cf5cc76edd (DEFAULT_TABLE table=default.Table_2ddc2d9b219945ae96fba0f3c312f2ea [id=6ac90f0f8ac84ed5953d37156d9b768b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.947333 28382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c75278db2234b11bf95f1cf5cc76edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.956084 28800 raft_consensus.cc:2272] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.972955 29055 raft_consensus.cc:493] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:23.973129 29055 raft_consensus.cc:515] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:23.973863 29055 leader_election.cc:290] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:23.975217 28484 leader_election.cc:343] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 1d23b5d2d67945a6ab9591969acb0d27
I20260618 13:59:23.975436 28801 ts_tablet_manager.cc:1929] T d6e3060118e844559803a649224d7474 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:23.975627 28801 log.cc:1199] T d6e3060118e844559803a649224d7474 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/d6e3060118e844559803a649224d7474
I20260618 13:59:23.976126 28801 ts_tablet_manager.cc:1950] T d6e3060118e844559803a649224d7474 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:23.976656 28801 tablet_replica.cc:333] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:23.976926 28801 raft_consensus.cc:2243] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:23.979415 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet d6e3060118e844559803a649224d7474 (table default.Table_b35e2c7c7707422d93364bf9be3bab3f [id=625859c44f3246eaacad14ba1f027d9c]) successfully deleted
I20260618 13:59:23.980012 28652 tablet_service.cc:1511] Processing CreateTablet for tablet be59f988f0e34b9480f859cf88b9c223 (DEFAULT_TABLE table=default.Table_b8741a7fd29f4e78a948b1c419595a85 [id=6f4d908593ae40f09e2c94cef218e1fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.984817 28800 ts_tablet_manager.cc:1916] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.985956 28652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be59f988f0e34b9480f859cf88b9c223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.988713 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_b583e9f1f2464ebe9df9b4663bb0a4f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:23.989413 28229 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_cd647e83c17a49219bc44d240cb9fb66" } new_table_name: "default.Table_04faa8df2e7c49549ccf29cfca0905fa" modify_external_catalogs: true
I20260618 13:59:23.989318 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_ff3779f2af2443428f9c1e3772091659"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:23.990104 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff3779f2af2443428f9c1e3772091659 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.990844 28225 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_85f1769dbcad44ce9a4f98a5493c1530" } new_table_name: "default.Table_3deacb2c9668491a85bbb09ae3f57d66" modify_external_catalogs: true
I20260618 13:59:23.991703 28801 raft_consensus.cc:2272] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:23.991813 28661 tablet_service.cc:1511] Processing CreateTablet for tablet 5c75278db2234b11bf95f1cf5cc76edd (DEFAULT_TABLE table=default.Table_2ddc2d9b219945ae96fba0f3c312f2ea [id=6ac90f0f8ac84ed5953d37156d9b768b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.992013 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_0a85d79af65f48a3803bfd1248593a11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:23.992295 28801 ts_tablet_manager.cc:1916] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:23.992373 28661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c75278db2234b11bf95f1cf5cc76edd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:23.992434 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a85d79af65f48a3803bfd1248593a11 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:23.993424 28224 master_service.cc:960] ReplaceTablet: received request to replace tablet f89c9954d48e407c96c294c26644e25c from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:23.994974 28380 tablet_service.cc:1511] Processing CreateTablet for tablet be59f988f0e34b9480f859cf88b9c223 (DEFAULT_TABLE table=default.Table_b8741a7fd29f4e78a948b1c419595a85 [id=6f4d908593ae40f09e2c94cef218e1fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.995576 28380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be59f988f0e34b9480f859cf88b9c223. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:23.997036 28483 leader_election.cc:343] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 1d23b5d2d67945a6ab9591969acb0d27
I20260618 13:59:23.997159 28483 leader_election.cc:304] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:23.997357 28483 raft_consensus.cc:2686] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260618 13:59:23.998713 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (DEFAULT_TABLE table=default.Table_768d783dc0a7416686369250c3cd855c [id=0321ff97d1674a3891eae8cbba76387a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.999329 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a9bd8577f4f4a9593a3ffeab582b02e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:23.999317 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_ec169b1f1c0b4377bbd3eb6df8dcab96"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:23.999850 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec169b1f1c0b4377bbd3eb6df8dcab96 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.000512 28527 tablet_service.cc:1511] Processing CreateTablet for tablet be59f988f0e34b9480f859cf88b9c223 (DEFAULT_TABLE table=default.Table_b8741a7fd29f4e78a948b1c419595a85 [id=6f4d908593ae40f09e2c94cef218e1fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.000910 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be59f988f0e34b9480f859cf88b9c223. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:24.005797 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b583e9f1f2464ebe9df9b4663bb0a4f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.006316 28653 tablet_service.cc:1511] Processing CreateTablet for tablet 8fd9fa9675234db2abff694a517c0ea8 (DEFAULT_TABLE table=default.Table_726d3825b282496fb87c67a8975e3fcc [id=f914bcce8551443caa218e7a92967307]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.007412 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 92ddbdfdfff547588a03f453e413af8f (DEFAULT_TABLE table=default.Table_93b6c79a288040ceae924af14f939c35 [id=61ce711b673b44189a507d1abf26bd75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.978452 28522 tablet_service.cc:1511] Processing CreateTablet for tablet 5c75278db2234b11bf95f1cf5cc76edd (DEFAULT_TABLE table=default.Table_2ddc2d9b219945ae96fba0f3c312f2ea [id=6ac90f0f8ac84ed5953d37156d9b768b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:23.983223 28596 heartbeater.cc:344] Connected to a master server at 127.27.100.61:45967
I20260618 13:59:24.008797 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92ddbdfdfff547588a03f453e413af8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.011307 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fd9fa9675234db2abff694a517c0ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.000134 28655 tablet_service.cc:1511] Processing CreateTablet for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (DEFAULT_TABLE table=default.Table_768d783dc0a7416686369250c3cd855c [id=0321ff97d1674a3891eae8cbba76387a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.016750 28651 tablet_service.cc:1511] Processing CreateTablet for tablet 04dac931e62a448d99633878eb70b92f (DEFAULT_TABLE table=default.Table_f37d21a9a59249d6a0b87c3788ff4873 [id=22eb9037fc66472793e9fd3f03fd7218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.017874 28521 tablet_service.cc:1511] Processing CreateTablet for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (DEFAULT_TABLE table=default.Table_768d783dc0a7416686369250c3cd855c [id=0321ff97d1674a3891eae8cbba76387a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.020895 28799 raft_consensus.cc:359] T 68b8293a7c924a449307ab88a1cf9290 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.021193 28799 raft_consensus.cc:385] T 68b8293a7c924a449307ab88a1cf9290 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.021839 28799 raft_consensus.cc:740] T 68b8293a7c924a449307ab88a1cf9290 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.022178 28799 consensus_queue.cc:260] T 68b8293a7c924a449307ab88a1cf9290 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.023880 28385 tablet_service.cc:1511] Processing CreateTablet for tablet 04dac931e62a448d99633878eb70b92f (DEFAULT_TABLE table=default.Table_f37d21a9a59249d6a0b87c3788ff4873 [id=22eb9037fc66472793e9fd3f03fd7218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.024441 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04dac931e62a448d99633878eb70b92f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.025063 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a9bd8577f4f4a9593a3ffeab582b02e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.025863 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04dac931e62a448d99633878eb70b92f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.028766 28518 tablet_service.cc:1511] Processing CreateTablet for tablet 04dac931e62a448d99633878eb70b92f (DEFAULT_TABLE table=default.Table_f37d21a9a59249d6a0b87c3788ff4873 [id=22eb9037fc66472793e9fd3f03fd7218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.031522 28798 tablet_bootstrap.cc:492] T ebf502f25e9b4342bfaa58f42db7f2f5 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:24.031714 28798 ts_tablet_manager.cc:1403] T ebf502f25e9b4342bfaa58f42db7f2f5 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.256s user 0.035s sys 0.046s
I20260618 13:59:24.032698 28798 raft_consensus.cc:359] T ebf502f25e9b4342bfaa58f42db7f2f5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.032939 28798 raft_consensus.cc:385] T ebf502f25e9b4342bfaa58f42db7f2f5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.033020 28798 raft_consensus.cc:740] T ebf502f25e9b4342bfaa58f42db7f2f5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.033295 28798 consensus_queue.cc:260] T ebf502f25e9b4342bfaa58f42db7f2f5 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.034112 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c75278db2234b11bf95f1cf5cc76edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.042057 28515 tablet_service.cc:1511] Processing CreateTablet for tablet 92ddbdfdfff547588a03f453e413af8f (DEFAULT_TABLE table=default.Table_93b6c79a288040ceae924af14f939c35 [id=61ce711b673b44189a507d1abf26bd75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.042838 28222 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_c97dcabd4c2b45a18f445a099ec7ace9" } modify_external_catalogs: true
I20260618 13:59:24.043062 28222 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_c97dcabd4c2b45a18f445a099ec7ace9" } modify_external_catalogs: true
I20260618 13:59:24.044471 28226 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_d81b926f20864cfe9915d642c53fa6f2" } modify_external_catalogs: true
I20260618 13:59:24.044616 28226 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_d81b926f20864cfe9915d642c53fa6f2" } modify_external_catalogs: true
I20260618 13:59:24.045828 28227 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_ddcc7a2b28ca4095be58d4c859fd964a" } modify_external_catalogs: true
I20260618 13:59:24.045996 28227 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_ddcc7a2b28ca4095be58d4c859fd964a" } modify_external_catalogs: true
I20260618 13:59:24.040993 28799 ts_tablet_manager.cc:1434] T 68b8293a7c924a449307ab88a1cf9290 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.148s user 0.000s sys 0.002s
I20260618 13:59:24.023005 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 92ddbdfdfff547588a03f453e413af8f (DEFAULT_TABLE table=default.Table_93b6c79a288040ceae924af14f939c35 [id=61ce711b673b44189a507d1abf26bd75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.045190 28381 tablet_service.cc:1511] Processing CreateTablet for tablet d1baf86497d442dbbbddbc3498bebbee (DEFAULT_TABLE table=default.Table_dac9c04f74664cce9c065edc3e491bf6 [id=5fe80671a240424da5ef1d7958caba2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.045634 28384 tablet_service.cc:1511] Processing CreateTablet for tablet 450c68da26404767b01c5e8da3f6286d (DEFAULT_TABLE table=default.Table_ea7b830f69254dc49cbc9e3c17c3be85 [id=de842622055849ceac0bac11fbcba6e8]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:24.053391 28229 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2f3d2f236fd44ab8af0524ea282cfa94 consensus state has no leader
I20260618 13:59:24.053853 28229 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_7699960fdee24242bf51bcfcbd6f68c2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.054458 28229 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7699960fdee24242bf51bcfcbd6f68c2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:24.055824 28225 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ab521028ac1a4d0da013112d76bb169f has no consensus state
I20260618 13:59:24.056197 28225 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_caa35bd6fefa4c5bab90b4b7f1ec48d9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.056696 28225 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_caa35bd6fefa4c5bab90b4b7f1ec48d9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.063377 28391 tablet_service.cc:1511] Processing CreateTablet for tablet bce7bf74b6d24d0bb43808b8a62cf1b7 (DEFAULT_TABLE table=default.Table_3cdafadde7c146d38bdb87c358269c35 [id=214e8abbe81c4ff19009a4902bdef469]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.063378 28381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1baf86497d442dbbbddbc3498bebbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.068217 28223 master_service.cc:960] ReplaceTablet: received request to replace tablet eead369de58f40378254697cbbbc6958 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:24.069394 28224 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet f89c9954d48e407c96c294c26644e25c
I20260618 13:59:24.069892 28224 catalog_manager.cc:6784] ReplaceTablet: tablet f89c9954d48e407c96c294c26644e25c deleted and replaced by tablet b0d3d363766546c4a97b0763bad11f71
I20260618 13:59:24.070192 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_7a1ed16f3c984aebab144d2c9efe5e23"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.070787 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a1ed16f3c984aebab144d2c9efe5e23 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.072269 28221 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_c5c65f038c8f4f2783a705ffcc320629" } modify_external_catalogs: true
I20260618 13:59:24.072456 28221 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_c5c65f038c8f4f2783a705ffcc320629" } modify_external_catalogs: true
I20260618 13:59:24.073917 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_b5feee88c56b42b6bd1b8ecb81a134bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.074466 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5feee88c56b42b6bd1b8ecb81a134bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.074757 29086 raft_consensus.cc:740] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a950ab7cd66b4cb0b29592aa692c2dd0, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.077641 29086 consensus_queue.cc:260] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:24.078173 28226 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 08a2dee3ac5a4bec96445a6001717eda
I20260618 13:59:24.078496 28649 tablet_service.cc:1511] Processing CreateTablet for tablet bce7bf74b6d24d0bb43808b8a62cf1b7 (DEFAULT_TABLE table=default.Table_3cdafadde7c146d38bdb87c358269c35 [id=214e8abbe81c4ff19009a4902bdef469]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.078866 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_d6f2768172e8455882ce01051fc835a4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.079351 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d6f2768172e8455882ce01051fc835a4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.080005 28649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bce7bf74b6d24d0bb43808b8a62cf1b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.082235 28227 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet b0c9d4131ea24809ac081e7a73a21e1c
I20260618 13:59:24.083076 28222 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 68b8293a7c924a449307ab88a1cf9290
I20260618 13:59:24.087105 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet f89c9954d48e407c96c294c26644e25c with delete_type TABLET_DATA_DELETED (replaced by tablet b0d3d363766546c4a97b0763bad11f71) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:24.087306 28663 tablet_service.cc:1511] Processing CreateTablet for tablet 2f5a334d18ba4c06bb358fa86d8af80d (DEFAULT_TABLE table=default.Table_48fb6a23c02e4c7788107c2fcdcfdd57 [id=f90028f13e704830a408df1907fdae49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.087728 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f5a334d18ba4c06bb358fa86d8af80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.090605 29086 sys_catalog.cc:565] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 13:59:24.091359 28798 ts_tablet_manager.cc:1434] T ebf502f25e9b4342bfaa58f42db7f2f5 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.060s user 0.001s sys 0.002s
I20260618 13:59:24.091686 28798 tablet_bootstrap.cc:492] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:24.092003 28515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92ddbdfdfff547588a03f453e413af8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.092345 28521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a9bd8577f4f4a9593a3ffeab582b02e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.092573 29097 sys_catalog.cc:455] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:24.093761 29097 sys_catalog.cc:458] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 13:59:24.100310 28798 tablet_bootstrap.cc:654] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.100239 28662 tablet_service.cc:1511] Processing CreateTablet for tablet d1baf86497d442dbbbddbc3498bebbee (DEFAULT_TABLE table=default.Table_dac9c04f74664cce9c065edc3e491bf6 [id=5fe80671a240424da5ef1d7958caba2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.101552 28518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04dac931e62a448d99633878eb70b92f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.101655 29057 raft_consensus.cc:493] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:24.101770 29057 raft_consensus.cc:515] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.102401 29057 leader_election.cc:290] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:24.103519 28648 tablet_service.cc:1511] Processing CreateTablet for tablet 450c68da26404767b01c5e8da3f6286d (DEFAULT_TABLE table=default.Table_ea7b830f69254dc49cbc9e3c17c3be85 [id=de842622055849ceac0bac11fbcba6e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.102532 28810 raft_consensus.cc:493] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:24.103817 28810 raft_consensus.cc:515] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.104318 28810 leader_election.cc:290] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:24.104522 28682 raft_consensus.cc:3060] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:24.103075 28516 tablet_service.cc:1511] Processing CreateTablet for tablet bce7bf74b6d24d0bb43808b8a62cf1b7 (DEFAULT_TABLE table=default.Table_3cdafadde7c146d38bdb87c358269c35 [id=214e8abbe81c4ff19009a4902bdef469]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.104898 28516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bce7bf74b6d24d0bb43808b8a62cf1b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.103565 28511 tablet_service.cc:1558] Processing DeleteTablet for tablet f89c9954d48e407c96c294c26644e25c with delete_type TABLET_DATA_DELETED (replaced by tablet b0d3d363766546c4a97b0763bad11f71) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:24.111382 28484 leader_election.cc:343] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 5332b6bb76ea419086341738fd135b24
I20260618 13:59:24.112573 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8389b77e6b8407db38e07e85f7509b1" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:24.113191 28484 leader_election.cc:343] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:24.114284 28682 raft_consensus.cc:1275] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Refusing update from remote peer 97859748af2c41879ba2a668dda999d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 13:59:24.114722 28648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 450c68da26404767b01c5e8da3f6286d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.118078 28522 tablet_service.cc:1511] Processing CreateTablet for tablet d1baf86497d442dbbbddbc3498bebbee (DEFAULT_TABLE table=default.Table_dac9c04f74664cce9c065edc3e491bf6 [id=5fe80671a240424da5ef1d7958caba2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.119000 28810 consensus_queue.cc:1048] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:24.118964 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1baf86497d442dbbbddbc3498bebbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.127807 28527 tablet_service.cc:1511] Processing CreateTablet for tablet 450c68da26404767b01c5e8da3f6286d (DEFAULT_TABLE table=default.Table_ea7b830f69254dc49cbc9e3c17c3be85 [id=de842622055849ceac0bac11fbcba6e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.131368 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 450c68da26404767b01c5e8da3f6286d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.135720 28730 heartbeater.cc:344] Connected to a master server at 127.27.100.61:45967
I20260618 13:59:24.135946 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1baf86497d442dbbbddbc3498bebbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.139279 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet f89c9954d48e407c96c294c26644e25c with delete_type TABLET_DATA_DELETED (replaced by tablet b0d3d363766546c4a97b0763bad11f71) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.139467 28388 tablet_service.cc:1511] Processing CreateTablet for tablet 2f5a334d18ba4c06bb358fa86d8af80d (DEFAULT_TABLE table=default.Table_48fb6a23c02e4c7788107c2fcdcfdd57 [id=f90028f13e704830a408df1907fdae49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.141819 28410 raft_consensus.cc:1275] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Refusing update from remote peer 97859748af2c41879ba2a668dda999d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260618 13:59:24.143299 28483 leader_election.cc:343] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: b8389b77e6b8407db38e07e85f7509b1
I20260618 13:59:24.143461 28483 leader_election.cc:304] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:24.143864 28483 leader_election.cc:343] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 5332b6bb76ea419086341738fd135b24
I20260618 13:59:24.143996 28483 leader_election.cc:304] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:24.144558 29055 raft_consensus.cc:2749] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:24.144767 29055 raft_consensus.cc:2749] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:24.144912 29055 consensus_queue.cc:1048] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:24.147429 28799 tablet_bootstrap.cc:492] T fca76774f119435899188313a33dad98 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:24.147603 28801 ts_tablet_manager.cc:1929] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.147715 28801 log.cc:1199] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/1d23b5d2d67945a6ab9591969acb0d27
I20260618 13:59:24.148065 28801 ts_tablet_manager.cc:1950] T 1d23b5d2d67945a6ab9591969acb0d27 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:24.148478 28801 tablet_replica.cc:333] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:24.149334 28801 raft_consensus.cc:2243] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.149492 28801 raft_consensus.cc:2272] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.150022 28801 ts_tablet_manager.cc:1916] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.154202 28798 tablet_bootstrap.cc:492] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:24.154348 28798 ts_tablet_manager.cc:1403] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.063s user 0.002s sys 0.003s
I20260618 13:59:24.155215 28798 raft_consensus.cc:359] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.155388 28798 raft_consensus.cc:385] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.155478 28798 raft_consensus.cc:740] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.155746 28798 consensus_queue.cc:260] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.159847 28511 tablet_service.cc:1558] Processing DeleteTablet for tablet 68b8293a7c924a449307ab88a1cf9290 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.162106 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet 08a2dee3ac5a4bec96445a6001717eda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.163168 28516 tablet_service.cc:1558] Processing DeleteTablet for tablet b0c9d4131ea24809ac081e7a73a21e1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.166855 28518 tablet_service.cc:1511] Processing CreateTablet for tablet 2f5a334d18ba4c06bb358fa86d8af80d (DEFAULT_TABLE table=default.Table_48fb6a23c02e4c7788107c2fcdcfdd57 [id=f90028f13e704830a408df1907fdae49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.167339 28518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f5a334d18ba4c06bb358fa86d8af80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.170719 28798 ts_tablet_manager.cc:1434] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.016s user 0.002s sys 0.000s
I20260618 13:59:24.178524 28516 tablet_service.cc:1558] Processing DeleteTablet for tablet ebf502f25e9b4342bfaa58f42db7f2f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.179059 28798 tablet_bootstrap.cc:492] T d980cda1c60e4edca7b4d7537f4f5837 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:24.179979 28657 tablet_service.cc:1511] Processing CreateTablet for tablet c881dd3cb66344969310956ad6e83cc0 (DEFAULT_TABLE table=default.Table_d31dc25d118d4cd2bb4dd8f034752827 [id=b1d85ea3aa7047d285ab7c86da8213fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.181954 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 1d23b5d2d67945a6ab9591969acb0d27 (table default.Table_48dff8bdaa4e430b8850f8d688195bcf [id=fa4cd457fb154247acbfc842852eef04]) successfully deleted
I20260618 13:59:24.182799 28225 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_b49acca2ee444ece953fcedf4b6d28d7" } new_table_name: "default.Table_706eec6806a94d3f8d5b60708a0c1cda" modify_external_catalogs: true
I20260618 13:59:24.182859 28229 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_04faa8df2e7c49549ccf29cfca0905fa" } modify_external_catalogs: true
I20260618 13:59:24.183123 28229 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_04faa8df2e7c49549ccf29cfca0905fa" } modify_external_catalogs: true
I20260618 13:59:24.183797 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c881dd3cb66344969310956ad6e83cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.183928 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet b0c9d4131ea24809ac081e7a73a21e1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:24.184574 28227 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_2ddc2d9b219945ae96fba0f3c312f2ea" } new_table_name: "default.Table_9d7999641cc344f18ccc672c49518192" modify_external_catalogs: true
I20260618 13:59:24.186088 28222 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_48fb6a23c02e4c7788107c2fcdcfdd57" } new_table_name: "default.Table_2cb0539405824600bb10d2712fb17d31" modify_external_catalogs: true
I20260618 13:59:24.186548 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet 08a2dee3ac5a4bec96445a6001717eda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:24.186554 28516 tablet_service.cc:1511] Processing CreateTablet for tablet c881dd3cb66344969310956ad6e83cc0 (DEFAULT_TABLE table=default.Table_d31dc25d118d4cd2bb4dd8f034752827 [id=b1d85ea3aa7047d285ab7c86da8213fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.186957 28516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c881dd3cb66344969310956ad6e83cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.187229 28659 tablet_service.cc:1558] Processing DeleteTablet for tablet 68b8293a7c924a449307ab88a1cf9290 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:24.187420 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_142b5f36b1c5441595c29bf65d537a97"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.188319 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_142b5f36b1c5441595c29bf65d537a97 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:24.193495 28797 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.509s user 0.034s sys 0.038s
W20260618 13:59:24.193606 28797 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.509s user 0.034s sys 0.038s
I20260618 13:59:24.194603 28797 tablet_bootstrap.cc:492] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:24.194756 28797 ts_tablet_manager.cc:1403] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.515s user 0.036s sys 0.040s
I20260618 13:59:24.195333 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 08a2dee3ac5a4bec96445a6001717eda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.195600 28797 raft_consensus.cc:359] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.195796 28797 raft_consensus.cc:385] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.195899 28797 raft_consensus.cc:740] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.196193 28379 tablet_service.cc:1558] Processing DeleteTablet for tablet b0c9d4131ea24809ac081e7a73a21e1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.196141 28797 consensus_queue.cc:260] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.196738 28797 ts_tablet_manager.cc:1434] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:24.201920 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet 37fedd27109944e2b3c870418c117758 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.202606 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92ddbdfdfff547588a03f453e413af8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.202910 28521 tablet_service.cc:1511] Processing CreateTablet for tablet be00bfa9bc1a4b83b18bfad05ba1495e (DEFAULT_TABLE table=default.Table_37ae7dae52a34dbeb8420e8aa7529336 [id=abeeb5e876424a1cb99b2a09083c8230]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.203290 28378 tablet_service.cc:1558] Processing DeleteTablet for tablet 68b8293a7c924a449307ab88a1cf9290 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.203339 28521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be00bfa9bc1a4b83b18bfad05ba1495e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.207787 28377 tablet_service.cc:1511] Processing CreateTablet for tablet be00bfa9bc1a4b83b18bfad05ba1495e (DEFAULT_TABLE table=default.Table_37ae7dae52a34dbeb8420e8aa7529336 [id=abeeb5e876424a1cb99b2a09083c8230]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.210332 28798 tablet_bootstrap.cc:654] T d980cda1c60e4edca7b4d7537f4f5837 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.213013 28223 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet eead369de58f40378254697cbbbc6958
I20260618 13:59:24.213501 28223 catalog_manager.cc:6784] ReplaceTablet: tablet eead369de58f40378254697cbbbc6958 deleted and replaced by tablet d464dd402d2044469e364ed37fdfb39b
I20260618 13:59:24.213711 28223 master_service.cc:960] ReplaceTablet: received request to replace tablet 125bb6885ab348d885d5a3b4bdd45bc7 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:24.214130 28223 master_service.cc:960] ReplaceTablet: received request to replace tablet 6eea4d59935d40aa97dc10ec7469aa56 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:24.215576 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bce7bf74b6d24d0bb43808b8a62cf1b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.222150 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet eead369de58f40378254697cbbbc6958 with delete_type TABLET_DATA_DELETED (replaced by tablet d464dd402d2044469e364ed37fdfb39b) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.227805 28221 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 516421cd843b44769efa2b555697c065
W20260618 13:59:24.230263 28227 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5c75278db2234b11bf95f1cf5cc76edd consensus state has no leader
W20260618 13:59:24.231261 28225 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1f81428f804a485c967026c4527fd250 consensus state has no leader
I20260618 13:59:24.232080 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet 516421cd843b44769efa2b555697c065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.233814 28801 ts_tablet_manager.cc:1929] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.240239 28225 master_service.cc:960] ReplaceTablet: received request to replace tablet 2188ebf1727240cd886ba7662f8fcf43 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:24.241917 28386 tablet_service.cc:1511] Processing CreateTablet for tablet c881dd3cb66344969310956ad6e83cc0 (DEFAULT_TABLE table=default.Table_d31dc25d118d4cd2bb4dd8f034752827 [id=b1d85ea3aa7047d285ab7c86da8213fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.242384 28378 tablet_service.cc:1558] Processing DeleteTablet for tablet 516421cd843b44769efa2b555697c065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.243086 28649 tablet_service.cc:1558] Processing DeleteTablet for tablet eead369de58f40378254697cbbbc6958 with delete_type TABLET_DATA_DELETED (replaced by tablet d464dd402d2044469e364ed37fdfb39b) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:24.235344 28798 tablet_bootstrap.cc:492] T d980cda1c60e4edca7b4d7537f4f5837 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:24.243861 28801 log.cc:1199] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/b8389b77e6b8407db38e07e85f7509b1
I20260618 13:59:24.243942 28798 ts_tablet_manager.cc:1403] T d980cda1c60e4edca7b4d7537f4f5837 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.065s user 0.004s sys 0.000s
I20260618 13:59:24.243494 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet 516421cd843b44769efa2b555697c065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:24.244449 28801 ts_tablet_manager.cc:1950] T b8389b77e6b8407db38e07e85f7509b1 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:24.244894 28798 raft_consensus.cc:359] T d980cda1c60e4edca7b4d7537f4f5837 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.245121 28798 raft_consensus.cc:385] T d980cda1c60e4edca7b4d7537f4f5837 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.245200 28798 raft_consensus.cc:740] T d980cda1c60e4edca7b4d7537f4f5837 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.245522 28797 tablet_bootstrap.cc:492] T ebf502f25e9b4342bfaa58f42db7f2f5 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:24.245600 28798 consensus_queue.cc:260] T d980cda1c60e4edca7b4d7537f4f5837 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.247617 28798 ts_tablet_manager.cc:1434] T d980cda1c60e4edca7b4d7537f4f5837 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:24.248310 28798 tablet_bootstrap.cc:492] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:24.249652 28797 tablet_bootstrap.cc:654] T ebf502f25e9b4342bfaa58f42db7f2f5 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.251130 28798 tablet_bootstrap.cc:654] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.251822 28596 heartbeater.cc:344] Connected to a master server at 127.27.100.61:45967
I20260618 13:59:24.256052 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet d980cda1c60e4edca7b4d7537f4f5837 with delete_type TABLET_DATA_DELETED (replaced by tablet 98b42d23832545ae81a7cdff19e4689f) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.257122 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet b8389b77e6b8407db38e07e85f7509b1 (table default.Table_a9703f9d3b54453691c354131f824e22 [id=f298b441bd9b444b9edd832be0adb03a]) successfully deleted
W20260618 13:59:24.258085 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet b16f7d3c7837498e996bba1818110642 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b16f7d3c7837498e996bba1818110642 already in progress: creating tablet
I20260618 13:59:24.261834 28225 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_7a1ed16f3c984aebab144d2c9efe5e23" } new_table_name: "default.Table_689e7dd93b784dc5bbded16bd6cba279" modify_external_catalogs: true
I20260618 13:59:24.261988 28226 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_5b3dc6cb553c45638fde791608253e3c" } new_table_name: "default.Table_8783a8d8c526428281436b62b2581a33" modify_external_catalogs: true
W20260618 13:59:24.262326 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 864f2602840c40fc8606b3784e671cba because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 864f2602840c40fc8606b3784e671cba already in progress: creating tablet
I20260618 13:59:24.262777 28801 tablet_replica.cc:333] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:24.263228 28801 raft_consensus.cc:2243] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260618 13:59:24.264330 28222 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2f5a334d18ba4c06bb358fa86d8af80d consensus state has no leader
I20260618 13:59:24.264386 28653 tablet_service.cc:1511] Processing CreateTablet for tablet be00bfa9bc1a4b83b18bfad05ba1495e (DEFAULT_TABLE table=default.Table_37ae7dae52a34dbeb8420e8aa7529336 [id=abeeb5e876424a1cb99b2a09083c8230]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.265688 28800 ts_tablet_manager.cc:1929] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:24.265837 28800 log.cc:1199] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/31b328882af549a78535cdd55c6ea09c
I20260618 13:59:24.266008 28801 raft_consensus.cc:2272] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.266319 28800 ts_tablet_manager.cc:1950] T 31b328882af549a78535cdd55c6ea09c P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:24.266968 28801 ts_tablet_manager.cc:1916] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.267680 28380 tablet_service.cc:1558] Processing DeleteTablet for tablet eead369de58f40378254697cbbbc6958 with delete_type TABLET_DATA_DELETED (replaced by tablet d464dd402d2044469e364ed37fdfb39b) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.270470 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c881dd3cb66344969310956ad6e83cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.271914 28377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be00bfa9bc1a4b83b18bfad05ba1495e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.275029 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_387aa217e7f04d2f826dc1f82c256a97"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.275624 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_387aa217e7f04d2f826dc1f82c256a97 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.275838 28802 ts_tablet_manager.cc:1929] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.275883 28797 tablet_bootstrap.cc:492] T ebf502f25e9b4342bfaa58f42db7f2f5 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:24.276060 28797 ts_tablet_manager.cc:1403] T ebf502f25e9b4342bfaa58f42db7f2f5 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.031s user 0.005s sys 0.001s
I20260618 13:59:24.276906 28797 raft_consensus.cc:359] T ebf502f25e9b4342bfaa58f42db7f2f5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.277118 28797 raft_consensus.cc:385] T ebf502f25e9b4342bfaa58f42db7f2f5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.277200 28797 raft_consensus.cc:740] T ebf502f25e9b4342bfaa58f42db7f2f5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.277637 28797 consensus_queue.cc:260] T ebf502f25e9b4342bfaa58f42db7f2f5 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.278165 28797 ts_tablet_manager.cc:1434] T ebf502f25e9b4342bfaa58f42db7f2f5 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:24.278209 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be00bfa9bc1a4b83b18bfad05ba1495e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.278510 28797 tablet_bootstrap.cc:492] T 37fedd27109944e2b3c870418c117758 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:24.282521 28458 heartbeater.cc:344] Connected to a master server at 127.27.100.61:45967
I20260618 13:59:24.282966 28797 tablet_bootstrap.cc:654] T 37fedd27109944e2b3c870418c117758 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.283327 28802 log.cc:1199] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/3752f49c9fc64071a45e0ce4ff4cfef2
I20260618 13:59:24.288959 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f5a334d18ba4c06bb358fa86d8af80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.288995 28798 tablet_bootstrap.cc:492] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:24.289281 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 450c68da26404767b01c5e8da3f6286d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.290582 28810 raft_consensus.cc:493] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:24.290750 28810 raft_consensus.cc:515] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.291486 28810 leader_election.cc:290] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:24.292670 28484 leader_election.cc:343] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 4ebed81a21b14db48ecb69299e0956cd
W20260618 13:59:24.293043 28483 leader_election.cc:343] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 4ebed81a21b14db48ecb69299e0956cd
I20260618 13:59:24.293170 28483 leader_election.cc:304] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:24.293682 28810 raft_consensus.cc:2749] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:24.294298 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 31b328882af549a78535cdd55c6ea09c (table default.Table_823368bbdf8242739a6022f65280f5a2 [id=28f3530b169a44eb9c262a718c8ea6d3]) successfully deleted
W20260618 13:59:24.294592 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 864f2602840c40fc8606b3784e671cba because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 864f2602840c40fc8606b3784e671cba already in progress: creating tablet
W20260618 13:59:24.294802 28215 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 4ebed81a21b14db48ecb69299e0956cd because the tablet was not found. No further retry: Not found: Tablet not found: 4ebed81a21b14db48ecb69299e0956cd
I20260618 13:59:24.295336 28800 tablet_replica.cc:333] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:24.295524 28800 raft_consensus.cc:2243] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.295723 28800 raft_consensus.cc:2272] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.296275 28800 ts_tablet_manager.cc:1916] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.299328 28802 ts_tablet_manager.cc:1950] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:24.299994 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 3752f49c9fc64071a45e0ce4ff4cfef2 (table default.Table_046616bed79749d5923cbd9ab9dff2b4 [id=af7cd415b54c4e4d8c7fb6f29dc8ff7b]) successfully deleted
I20260618 13:59:24.300371 28802 tablet_replica.cc:333] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:24.300601 28802 raft_consensus.cc:2243] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.301234 28801 ts_tablet_manager.cc:1929] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.302094 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_d4895ce143324c7f9c575472ce329ed3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.302731 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d4895ce143324c7f9c575472ce329ed3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.305585 28802 raft_consensus.cc:2272] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.309731 28806 raft_consensus.cc:493] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:24.309912 28806 raft_consensus.cc:515] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.310590 28806 leader_election.cc:290] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:24.310788 28224 master_service.cc:960] ReplaceTablet: received request to replace tablet 8fd9fa9675234db2abff694a517c0ea8 from {username='slave'} at 127.0.0.1:45138
W20260618 13:59:24.312187 28618 leader_election.cc:343] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Not found: Tablet not found: b8389b77e6b8407db38e07e85f7509b1
W20260618 13:59:24.316000 28617 leader_election.cc:343] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: b8389b77e6b8407db38e07e85f7509b1
I20260618 13:59:24.316128 28617 leader_election.cc:304] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:24.316761 28806 raft_consensus.cc:2749] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:24.317337 28802 ts_tablet_manager.cc:1916] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.322700 28801 log.cc:1199] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/5332b6bb76ea419086341738fd135b24
I20260618 13:59:24.323271 28799 tablet_bootstrap.cc:654] T fca76774f119435899188313a33dad98 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.323429 28801 ts_tablet_manager.cc:1950] T 5332b6bb76ea419086341738fd135b24 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:24.324566 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 5332b6bb76ea419086341738fd135b24 (table default.Table_e4cb8c97170c465980e9eeb3f6e4f88c [id=116d7007e8754aab8d37956bcb596f13]) successfully deleted
I20260618 13:59:24.325524 28801 tablet_replica.cc:333] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
W20260618 13:59:24.325976 28215 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet 36b8a46a41e543c58a946b6764618c7d (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 36b8a46a41e543c58a946b6764618c7d consensus state has no leader
I20260618 13:59:24.326349 28801 raft_consensus.cc:2243] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.326992 28801 raft_consensus.cc:2272] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.327981 28801 ts_tablet_manager.cc:1916] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.337314 28802 ts_tablet_manager.cc:1929] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.342872 28800 ts_tablet_manager.cc:1929] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.343024 28800 log.cc:1199] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/311ba4dd83984f379c8f4f8274bc26c1
I20260618 13:59:24.343488 28800 ts_tablet_manager.cc:1950] T 311ba4dd83984f379c8f4f8274bc26c1 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:24.344151 28802 log.cc:1199] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/e1a75029992645c5b942c0ec07862bb4
I20260618 13:59:24.344743 28802 ts_tablet_manager.cc:1950] T e1a75029992645c5b942c0ec07862bb4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:24.344846 28801 ts_tablet_manager.cc:1929] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.345499 28801 log.cc:1199] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/059716cec7f94ea6ad8989fecc6f832a
I20260618 13:59:24.346127 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet e1a75029992645c5b942c0ec07862bb4 (table default.Table_af2d1c7150e84f96807cda9b1dabd299 [id=bd90cbbd5592449480ef377189a9c2de]) successfully deleted
I20260618 13:59:24.347211 28801 ts_tablet_manager.cc:1950] T 059716cec7f94ea6ad8989fecc6f832a P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
W20260618 13:59:24.347424 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 0fae928833e44d2ba860d6184d4c0142 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0fae928833e44d2ba860d6184d4c0142 already in progress: creating tablet
I20260618 13:59:24.347761 28799 tablet_bootstrap.cc:492] T fca76774f119435899188313a33dad98 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:24.347940 28799 ts_tablet_manager.cc:1403] T fca76774f119435899188313a33dad98 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.211s user 0.006s sys 0.004s
W20260618 13:59:24.348255 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet bc0adedecee4405e86b0a37b9e62e352 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc0adedecee4405e86b0a37b9e62e352 already in progress: creating tablet
W20260618 13:59:24.348866 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 4c2b924a096b479cad31c29bf7c99172 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4c2b924a096b479cad31c29bf7c99172 already in progress: creating tablet
I20260618 13:59:24.348781 28799 raft_consensus.cc:359] T fca76774f119435899188313a33dad98 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.348987 28799 raft_consensus.cc:385] T fca76774f119435899188313a33dad98 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.349076 28799 raft_consensus.cc:740] T fca76774f119435899188313a33dad98 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.349332 28799 consensus_queue.cc:260] T fca76774f119435899188313a33dad98 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.349459 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 059716cec7f94ea6ad8989fecc6f832a (table default.Table_e0177aee52ca4680a5786ad73020ad75 [id=591e2be369c347e883141ed55927044d]) successfully deleted
W20260618 13:59:24.349601 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet f89c9954d48e407c96c294c26644e25c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f89c9954d48e407c96c294c26644e25c already in progress: creating tablet
I20260618 13:59:24.349821 28801 tablet_replica.cc:333] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:24.349937 28799 ts_tablet_manager.cc:1434] T fca76774f119435899188313a33dad98 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:24.350257 28799 tablet_bootstrap.cc:492] T 0a51e830e3014726b1d1c2608973f036 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:24.350272 28801 raft_consensus.cc:2243] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:24.351095 28801 pending_rounds.cc:70] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2: Trying to abort 1 pending ops.
I20260618 13:59:24.351735 28801 pending_rounds.cc:77] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298216606228594688 op_type: NO_OP noop_request { }
I20260618 13:59:24.352259 28801 raft_consensus.cc:2889] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:24.352609 28801 raft_consensus.cc:2272] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.353917 28801 ts_tablet_manager.cc:1916] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.357836 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 311ba4dd83984f379c8f4f8274bc26c1 (table default.Table_dbc49898f77e441cb501871dfae61db4 [id=d8ef984c664f4a6598c70a48e9aff826]) successfully deleted
I20260618 13:59:24.359597 28802 tablet_replica.cc:333] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:24.359895 28802 raft_consensus.cc:2243] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.360137 28802 raft_consensus.cc:2272] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.360678 28799 tablet_bootstrap.cc:654] T 0a51e830e3014726b1d1c2608973f036 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.363107 28802 ts_tablet_manager.cc:1916] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:24.363088 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 8b59255b14df4158afe37cb8890e7415 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b59255b14df4158afe37cb8890e7415 already in progress: creating tablet
W20260618 13:59:24.363351 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet e1a75029992645c5b942c0ec07862bb4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e1a75029992645c5b942c0ec07862bb4 already in progress: creating tablet
W20260618 13:59:24.363588 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet c84189a4b1194c87bbdf81f6e0ed56e4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c84189a4b1194c87bbdf81f6e0ed56e4 already in progress: creating tablet
W20260618 13:59:24.363828 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 3752f49c9fc64071a45e0ce4ff4cfef2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3752f49c9fc64071a45e0ce4ff4cfef2 already in progress: creating tablet
W20260618 13:59:24.364090 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 0fae928833e44d2ba860d6184d4c0142 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0fae928833e44d2ba860d6184d4c0142 already in progress: creating tablet
W20260618 13:59:24.364946 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet bc0adedecee4405e86b0a37b9e62e352 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc0adedecee4405e86b0a37b9e62e352 already in progress: creating tablet
W20260618 13:59:24.365247 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 4c2b924a096b479cad31c29bf7c99172 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4c2b924a096b479cad31c29bf7c99172 already in progress: creating tablet
W20260618 13:59:24.370904 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet f89c9954d48e407c96c294c26644e25c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f89c9954d48e407c96c294c26644e25c already in progress: creating tablet
W20260618 13:59:24.371423 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet b0c9d4131ea24809ac081e7a73a21e1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0c9d4131ea24809ac081e7a73a21e1c already in progress: creating tablet
W20260618 13:59:24.371645 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 08a2dee3ac5a4bec96445a6001717eda because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 08a2dee3ac5a4bec96445a6001717eda already in progress: creating tablet
W20260618 13:59:24.371845 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 68b8293a7c924a449307ab88a1cf9290 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68b8293a7c924a449307ab88a1cf9290 already in progress: creating tablet
W20260618 13:59:24.372026 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet eead369de58f40378254697cbbbc6958 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eead369de58f40378254697cbbbc6958 already in progress: creating tablet
I20260618 13:59:24.372438 29118 tablet.cc:1724] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:24.366644 28798 ts_tablet_manager.cc:1403] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.118s user 0.001s sys 0.005s
I20260618 13:59:24.374977 28798 raft_consensus.cc:359] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.375253 28798 raft_consensus.cc:385] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.375381 28798 raft_consensus.cc:740] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
W20260618 13:59:24.375617 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 516421cd843b44769efa2b555697c065 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 516421cd843b44769efa2b555697c065 already in progress: creating tablet
I20260618 13:59:24.376024 28798 consensus_queue.cc:260] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.382498 28798 ts_tablet_manager.cc:1434] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260618 13:59:24.383275 28228 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 411b0c333ae7450f9ada90afee553615 (table default.Table_5d0d3ddc1f4a4badbb853268cb2bba65 [id=cd89063e7d294d3fbb3b58358e781c90]). Expected version 2 got 0
I20260618 13:59:24.385228 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_1b32156053d845a5b192d2ccf583cac6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.385792 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1b32156053d845a5b192d2ccf583cac6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.387907 28221 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 411b0c333ae7450f9ada90afee553615 (table default.Table_5d0d3ddc1f4a4badbb853268cb2bba65 [id=cd89063e7d294d3fbb3b58358e781c90]). Expected version 2 got 0
I20260618 13:59:24.388605 28221 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_2cb0539405824600bb10d2712fb17d31" } modify_external_catalogs: true
I20260618 13:59:24.388803 28221 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_2cb0539405824600bb10d2712fb17d31" } modify_external_catalogs: true
I20260618 13:59:24.390302 28802 ts_tablet_manager.cc:1929] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.390839 28798 tablet_bootstrap.cc:492] T 1f0109536c24485c9934193f2ef8aeae P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:24.392192 28223 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 6eea4d59935d40aa97dc10ec7469aa56
I20260618 13:59:24.392710 28223 catalog_manager.cc:6784] ReplaceTablet: tablet 6eea4d59935d40aa97dc10ec7469aa56 deleted and replaced by tablet 4bc87770baa34842a25ed97a19354446
I20260618 13:59:24.393059 28223 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_191169025bff42cab93cf42503fa7220" } modify_external_catalogs: true
I20260618 13:59:24.393225 28223 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_191169025bff42cab93cf42503fa7220" } modify_external_catalogs: true
I20260618 13:59:24.394335 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 6eea4d59935d40aa97dc10ec7469aa56 with delete_type TABLET_DATA_DELETED (replaced by tablet 4bc87770baa34842a25ed97a19354446) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:24.395129 28225 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c5d046e0421142cd84c5b12d6a434f53 has no consensus state
I20260618 13:59:24.395514 28225 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_4f1453c90c9f4b49b95607b12ad0bb65" } modify_external_catalogs: true
I20260618 13:59:24.395685 28225 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_4f1453c90c9f4b49b95607b12ad0bb65" } modify_external_catalogs: true
I20260618 13:59:24.395797 28802 log.cc:1199] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/08a2dee3ac5a4bec96445a6001717eda
I20260618 13:59:24.396171 28802 ts_tablet_manager.cc:1950] T 08a2dee3ac5a4bec96445a6001717eda P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:24.396715 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 08a2dee3ac5a4bec96445a6001717eda (table default.Table_d81b926f20864cfe9915d642c53fa6f2 [id=80c0bb8d7847432783cb5c6c0bf76633]) successfully deleted
I20260618 13:59:24.397277 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 6eea4d59935d40aa97dc10ec7469aa56 with delete_type TABLET_DATA_DELETED (replaced by tablet 4bc87770baa34842a25ed97a19354446) from {username='slave'} at 127.0.0.1:43976
W20260618 13:59:24.397948 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 6eea4d59935d40aa97dc10ec7469aa56 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6eea4d59935d40aa97dc10ec7469aa56 already in progress: creating tablet
I20260618 13:59:24.398617 28802 tablet_replica.cc:333] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:24.399118 28799 tablet_bootstrap.cc:492] T 0a51e830e3014726b1d1c2608973f036 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:24.399312 28799 ts_tablet_manager.cc:1403] T 0a51e830e3014726b1d1c2608973f036 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.049s user 0.004s sys 0.000s
I20260618 13:59:24.399096 28802 raft_consensus.cc:2243] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.399597 28802 raft_consensus.cc:2272] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.400110 28802 ts_tablet_manager.cc:1916] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.401700 28798 tablet_bootstrap.cc:654] T 1f0109536c24485c9934193f2ef8aeae P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.400131 28799 raft_consensus.cc:359] T 0a51e830e3014726b1d1c2608973f036 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.402465 28799 raft_consensus.cc:385] T 0a51e830e3014726b1d1c2608973f036 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.402577 28799 raft_consensus.cc:740] T 0a51e830e3014726b1d1c2608973f036 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.402827 28799 consensus_queue.cc:260] T 0a51e830e3014726b1d1c2608973f036 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.403316 28799 ts_tablet_manager.cc:1434] T 0a51e830e3014726b1d1c2608973f036 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260618 13:59:24.403585 28799 tablet_bootstrap.cc:492] T 125bb6885ab348d885d5a3b4bdd45bc7 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:24.405194 28799 tablet_bootstrap.cc:654] T 125bb6885ab348d885d5a3b4bdd45bc7 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:24.408099 28226 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f278d0acbb0c4a0ca61640bc142e6335 consensus state has no leader
I20260618 13:59:24.408492 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_a8705c5cd49a40049ff1020a01aa7f5b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.409045 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a8705c5cd49a40049ff1020a01aa7f5b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.413050 29114 tablet.cc:1724] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:24.416529 29116 tablet.cc:1724] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:24.417870 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet ffd7d5ebaa94416a8eaf3e61bed945e1 with delete_type TABLET_DATA_DELETED (replaced by tablet d6871232860b41d489e34d95eb897241) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.425298 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet 6eea4d59935d40aa97dc10ec7469aa56 with delete_type TABLET_DATA_DELETED (replaced by tablet 4bc87770baa34842a25ed97a19354446) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.428158 28730 heartbeater.cc:344] Connected to a master server at 127.27.100.61:45967
I20260618 13:59:24.429862 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_c2afd6643b7b49b7992384135658ddfc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.430435 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c2afd6643b7b49b7992384135658ddfc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.433406 28224 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 8fd9fa9675234db2abff694a517c0ea8
I20260618 13:59:24.433948 28224 catalog_manager.cc:6784] ReplaceTablet: tablet 8fd9fa9675234db2abff694a517c0ea8 deleted and replaced by tablet 538077e459a44ed78c95f2e881b6e0b0
I20260618 13:59:24.435878 28223 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet aef7c1f111634c98af046e73fa9ffdeb
I20260618 13:59:24.434279 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_da651ce781664b8a8dfe3f1ab5774740"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.437376 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_da651ce781664b8a8dfe3f1ab5774740 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.439066 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet aef7c1f111634c98af046e73fa9ffdeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.439399 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet 8fd9fa9675234db2abff694a517c0ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet 538077e459a44ed78c95f2e881b6e0b0) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.439502 28802 ts_tablet_manager.cc:1929] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.440119 28802 log.cc:1199] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/b0c9d4131ea24809ac081e7a73a21e1c
I20260618 13:59:24.440959 28802 ts_tablet_manager.cc:1950] T b0c9d4131ea24809ac081e7a73a21e1c P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:24.443295 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet b0c9d4131ea24809ac081e7a73a21e1c (table default.Table_ddcc7a2b28ca4095be58d4c859fd964a [id=0bd492e029e346f68f2db501d798ace1]) successfully deleted
I20260618 13:59:24.445176 28802 tablet_replica.cc:333] T 68b8293a7c924a449307ab88a1cf9290 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:24.446357 28223 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 411b0c333ae7450f9ada90afee553615 (table default.Table_5d0d3ddc1f4a4badbb853268cb2bba65 [id=cd89063e7d294d3fbb3b58358e781c90]). Expected version 2 got 0
I20260618 13:59:24.446878 28802 raft_consensus.cc:2243] T 68b8293a7c924a449307ab88a1cf9290 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.447077 28802 raft_consensus.cc:2272] T 68b8293a7c924a449307ab88a1cf9290 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.447654 28802 ts_tablet_manager.cc:1916] T 68b8293a7c924a449307ab88a1cf9290 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.462348 28802 ts_tablet_manager.cc:1929] T 68b8293a7c924a449307ab88a1cf9290 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.470101 28802 log.cc:1199] T 68b8293a7c924a449307ab88a1cf9290 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/68b8293a7c924a449307ab88a1cf9290
I20260618 13:59:24.470934 28802 ts_tablet_manager.cc:1950] T 68b8293a7c924a449307ab88a1cf9290 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:24.475049 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 8fd9fa9675234db2abff694a517c0ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet 538077e459a44ed78c95f2e881b6e0b0) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:24.475354 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet 8fd9fa9675234db2abff694a517c0ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet 538077e459a44ed78c95f2e881b6e0b0) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.475695 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 68b8293a7c924a449307ab88a1cf9290 (table default.Table_c97dcabd4c2b45a18f445a099ec7ace9 [id=38bde7a301814aa285e4bd30bcc527aa]) successfully deleted
I20260618 13:59:24.475991 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet aef7c1f111634c98af046e73fa9ffdeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.476941 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet aef7c1f111634c98af046e73fa9ffdeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43976
W20260618 13:59:24.477259 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 516421cd843b44769efa2b555697c065 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 516421cd843b44769efa2b555697c065 already in progress: creating tablet
W20260618 13:59:24.478578 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet eead369de58f40378254697cbbbc6958 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eead369de58f40378254697cbbbc6958 already in progress: creating tablet
W20260618 13:59:24.478829 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 6eea4d59935d40aa97dc10ec7469aa56 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6eea4d59935d40aa97dc10ec7469aa56 already in progress: creating tablet
W20260618 13:59:24.479280 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet aef7c1f111634c98af046e73fa9ffdeb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aef7c1f111634c98af046e73fa9ffdeb already in progress: creating tablet
W20260618 13:59:24.479523 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 8fd9fa9675234db2abff694a517c0ea8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8fd9fa9675234db2abff694a517c0ea8 already in progress: creating tablet
W20260618 13:59:24.486907 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 8fd9fa9675234db2abff694a517c0ea8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8fd9fa9675234db2abff694a517c0ea8 already in progress: creating tablet
W20260618 13:59:24.487233 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet aef7c1f111634c98af046e73fa9ffdeb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aef7c1f111634c98af046e73fa9ffdeb already in progress: creating tablet
I20260618 13:59:24.489483 28220 catalog_manager.cc:6241] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: default.Table_5d0d3ddc1f4a4badbb853268cb2bba65 [id=cd89063e7d294d3fbb3b58358e781c90] alter complete (version 2)
I20260618 13:59:24.490310 28220 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_fdff0afd87584a53a884c9b76926d6d0" } modify_external_catalogs: true
I20260618 13:59:24.490561 28220 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_fdff0afd87584a53a884c9b76926d6d0" } modify_external_catalogs: true
I20260618 13:59:24.492815 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_7cd820b0428a41ada19a2f14560a67e4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:24.494096 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_8e7925d0e7eb421685f09b8c53667351"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.494657 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8e7925d0e7eb421685f09b8c53667351 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.496244 28222 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_c940d6e3d5d34adfaf39c7b5b60044e1" } new_table_name: "default.Table_26e7d8c4f91144268a2a43e83992667d" modify_external_catalogs: true
I20260618 13:59:24.497581 28224 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_37ae7dae52a34dbeb8420e8aa7529336" } new_table_name: "default.Table_330e421310fd41adba536e77619ab0ad" modify_external_catalogs: true
I20260618 13:59:24.498729 28225 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet fddea8fba4da40d7a5780e6750caf5e9
I20260618 13:59:24.499429 28225 master_service.cc:960] ReplaceTablet: received request to replace tablet e95c88e4b0a046a3ad69ee49af47c7c3 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:24.499686 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet fca76774f119435899188313a33dad98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.499958 28802 tablet_replica.cc:333] T fca76774f119435899188313a33dad98 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:24.499933 28225 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_ec169b1f1c0b4377bbd3eb6df8dcab96" } new_table_name: "default.Table_8c1fdd54393c4ad383e72a07837ead2a" modify_external_catalogs: true
I20260618 13:59:24.500532 28802 raft_consensus.cc:2243] T fca76774f119435899188313a33dad98 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.500732 28802 raft_consensus.cc:2272] T fca76774f119435899188313a33dad98 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.501283 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet fddea8fba4da40d7a5780e6750caf5e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.501353 28802 ts_tablet_manager.cc:1916] T fca76774f119435899188313a33dad98 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.502801 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_644a49408bd64f4d98eaa6650b3b93e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.503340 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_644a49408bd64f4d98eaa6650b3b93e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:24.505051 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7cd820b0428a41ada19a2f14560a67e4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.507448 28802 ts_tablet_manager.cc:1929] T fca76774f119435899188313a33dad98 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.507658 28802 log.cc:1199] T fca76774f119435899188313a33dad98 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/fca76774f119435899188313a33dad98
I20260618 13:59:24.508104 28802 ts_tablet_manager.cc:1950] T fca76774f119435899188313a33dad98 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:24.509198 28226 master_service.cc:960] ReplaceTablet: received request to replace tablet b51d0851efb34eaa9e9e4be6b292ef62 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:24.510138 28648 tablet_service.cc:1558] Processing DeleteTablet for tablet ebf502f25e9b4342bfaa58f42db7f2f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:24.510190 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet b8389b77e6b8407db38e07e85f7509b1 with delete_type TABLET_DATA_DELETED (replaced by tablet d9c00003300246ada2a356e3e1cf684b) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:24.510483 28800 tablet_replica.cc:333] T ebf502f25e9b4342bfaa58f42db7f2f5 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:24.510723 28800 raft_consensus.cc:2243] T ebf502f25e9b4342bfaa58f42db7f2f5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.510818 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet fca76774f119435899188313a33dad98 (table default.Table_4cf88aaddada4361837d7d49170f029b [id=50525d6257fa4de8a00dbee44d37f24d]) successfully deleted
I20260618 13:59:24.510932 28800 raft_consensus.cc:2272] T ebf502f25e9b4342bfaa58f42db7f2f5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:24.511047 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet fddea8fba4da40d7a5780e6750caf5e9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fddea8fba4da40d7a5780e6750caf5e9 already in progress: creating tablet
I20260618 13:59:24.511550 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a51e830e3014726b1d1c2608973f036 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.512010 28800 ts_tablet_manager.cc:1916] T ebf502f25e9b4342bfaa58f42db7f2f5 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.513115 28802 tablet_replica.cc:333] T 0a51e830e3014726b1d1c2608973f036 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:24.513348 28802 raft_consensus.cc:2243] T 0a51e830e3014726b1d1c2608973f036 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.513574 28802 raft_consensus.cc:2272] T 0a51e830e3014726b1d1c2608973f036 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.513846 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet fddea8fba4da40d7a5780e6750caf5e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:24.514293 28802 ts_tablet_manager.cc:1916] T 0a51e830e3014726b1d1c2608973f036 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.515153 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet fddea8fba4da40d7a5780e6750caf5e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.517534 28799 tablet_bootstrap.cc:492] T 125bb6885ab348d885d5a3b4bdd45bc7 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:24.517751 28799 ts_tablet_manager.cc:1403] T 125bb6885ab348d885d5a3b4bdd45bc7 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.114s user 0.014s sys 0.022s
I20260618 13:59:24.518258 28800 ts_tablet_manager.cc:1929] T ebf502f25e9b4342bfaa58f42db7f2f5 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.518478 28800 log.cc:1199] T ebf502f25e9b4342bfaa58f42db7f2f5 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/ebf502f25e9b4342bfaa58f42db7f2f5
I20260618 13:59:24.518853 28800 ts_tablet_manager.cc:1950] T ebf502f25e9b4342bfaa58f42db7f2f5 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:24.518823 28799 raft_consensus.cc:359] T 125bb6885ab348d885d5a3b4bdd45bc7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.519167 28799 raft_consensus.cc:385] T 125bb6885ab348d885d5a3b4bdd45bc7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.519282 28799 raft_consensus.cc:740] T 125bb6885ab348d885d5a3b4bdd45bc7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.519589 28810 raft_consensus.cc:493] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:24.519568 28799 consensus_queue.cc:260] T 125bb6885ab348d885d5a3b4bdd45bc7 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.519788 28810 raft_consensus.cc:515] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:24.521170 28483 leader_election.cc:343] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: ffd7d5ebaa94416a8eaf3e61bed945e1
I20260618 13:59:24.521171 28810 leader_election.cc:290] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:24.521775 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd7d5ebaa94416a8eaf3e61bed945e1" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:24.522452 28799 ts_tablet_manager.cc:1434] T 125bb6885ab348d885d5a3b4bdd45bc7 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
W20260618 13:59:24.523000 28484 leader_election.cc:343] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:24.523150 28484 leader_election.cc:304] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:24.523402 28799 tablet_bootstrap.cc:492] T 864f2602840c40fc8606b3784e671cba P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:24.524695 28810 raft_consensus.cc:2749] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:24.525187 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet ebf502f25e9b4342bfaa58f42db7f2f5 (table default.Table_c1cda1e770c24c7eb9ab6c1ad8a7e786 [id=19c30a68cf504a179bbb2a966b68445c]) successfully deleted
I20260618 13:59:24.526264 28798 tablet_bootstrap.cc:492] T 1f0109536c24485c9934193f2ef8aeae P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:24.526497 28798 ts_tablet_manager.cc:1403] T 1f0109536c24485c9934193f2ef8aeae P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.136s user 0.004s sys 0.000s
I20260618 13:59:24.526669 28800 tablet_replica.cc:333] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:24.526865 28800 raft_consensus.cc:2243] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.527037 28800 raft_consensus.cc:2272] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.528146 28800 ts_tablet_manager.cc:1916] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.529443 28802 ts_tablet_manager.cc:1929] T 0a51e830e3014726b1d1c2608973f036 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260618 13:59:24.530359 28222 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ece7909d056c4ba8a5dca6423f0c1781 consensus state has no leader
I20260618 13:59:24.532563 28222 master_service.cc:960] ReplaceTablet: received request to replace tablet 04dac931e62a448d99633878eb70b92f from {username='slave'} at 127.0.0.1:45138
W20260618 13:59:24.532936 28224 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet be00bfa9bc1a4b83b18bfad05ba1495e consensus state has no leader
W20260618 13:59:24.533316 28225 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3a0039a463ec4a9c82a21a3a69b565fc has no consensus state
I20260618 13:59:24.533911 28800 ts_tablet_manager.cc:1929] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.534022 28800 log.cc:1199] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/b8389b77e6b8407db38e07e85f7509b1
I20260618 13:59:24.534332 28800 ts_tablet_manager.cc:1950] T b8389b77e6b8407db38e07e85f7509b1 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:24.537984 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet b8389b77e6b8407db38e07e85f7509b1 (table default.Table_a9703f9d3b54453691c354131f824e22 [id=f298b441bd9b444b9edd832be0adb03a]) successfully deleted
I20260618 13:59:24.529989 28798 raft_consensus.cc:359] T 1f0109536c24485c9934193f2ef8aeae P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.530143 28799 tablet_bootstrap.cc:654] T 864f2602840c40fc8606b3784e671cba P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.538475 28798 raft_consensus.cc:385] T 1f0109536c24485c9934193f2ef8aeae P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.534741 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet 125bb6885ab348d885d5a3b4bdd45bc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.538581 28798 raft_consensus.cc:740] T 1f0109536c24485c9934193f2ef8aeae P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.538901 28798 consensus_queue.cc:260] T 1f0109536c24485c9934193f2ef8aeae P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.539605 28802 log.cc:1199] T 0a51e830e3014726b1d1c2608973f036 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/0a51e830e3014726b1d1c2608973f036
I20260618 13:59:24.540043 28802 ts_tablet_manager.cc:1950] T 0a51e830e3014726b1d1c2608973f036 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:24.541256 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 0a51e830e3014726b1d1c2608973f036 (table default.Table_696ead97d50747228517c265538b25fb [id=716ae493050c4288b9bf041c94523b05]) successfully deleted
I20260618 13:59:24.542303 28802 tablet_replica.cc:333] T 125bb6885ab348d885d5a3b4bdd45bc7 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:24.542728 28802 raft_consensus.cc:2243] T 125bb6885ab348d885d5a3b4bdd45bc7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.543184 28802 raft_consensus.cc:2272] T 125bb6885ab348d885d5a3b4bdd45bc7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.543802 28797 tablet_bootstrap.cc:492] T 37fedd27109944e2b3c870418c117758 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:24.543996 28797 ts_tablet_manager.cc:1403] T 37fedd27109944e2b3c870418c117758 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.266s user 0.005s sys 0.013s
I20260618 13:59:24.544896 28797 raft_consensus.cc:359] T 37fedd27109944e2b3c870418c117758 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.545133 28797 raft_consensus.cc:385] T 37fedd27109944e2b3c870418c117758 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.545506 28799 tablet_bootstrap.cc:492] T 864f2602840c40fc8606b3784e671cba P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:24.545522 28797 raft_consensus.cc:740] T 37fedd27109944e2b3c870418c117758 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.545706 28799 ts_tablet_manager.cc:1403] T 864f2602840c40fc8606b3784e671cba P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.003s
I20260618 13:59:24.545873 28797 consensus_queue.cc:260] T 37fedd27109944e2b3c870418c117758 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.549407 28797 ts_tablet_manager.cc:1434] T 37fedd27109944e2b3c870418c117758 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260618 13:59:24.551649 28797 tablet_bootstrap.cc:492] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:24.553393 28797 tablet_bootstrap.cc:654] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.553686 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f0109536c24485c9934193f2ef8aeae with delete_type TABLET_DATA_DELETED (replaced by tablet 6b8f273def3e4b7caa4d624cf87a106c) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:24.553869 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet fddea8fba4da40d7a5780e6750caf5e9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fddea8fba4da40d7a5780e6750caf5e9 already in progress: creating tablet
I20260618 13:59:24.554138 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 37fedd27109944e2b3c870418c117758 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:24.554451 28800 tablet_replica.cc:333] T 37fedd27109944e2b3c870418c117758 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:24.554512 28798 ts_tablet_manager.cc:1434] T 1f0109536c24485c9934193f2ef8aeae P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.025s user 0.002s sys 0.000s
I20260618 13:59:24.554674 28800 raft_consensus.cc:2243] T 37fedd27109944e2b3c870418c117758 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.554838 28800 raft_consensus.cc:2272] T 37fedd27109944e2b3c870418c117758 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.555570 28800 ts_tablet_manager.cc:1916] T 37fedd27109944e2b3c870418c117758 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.558913 28799 raft_consensus.cc:359] T 864f2602840c40fc8606b3784e671cba P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.559773 28799 raft_consensus.cc:385] T 864f2602840c40fc8606b3784e671cba P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.559935 28799 raft_consensus.cc:740] T 864f2602840c40fc8606b3784e671cba P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.560247 28799 consensus_queue.cc:260] T 864f2602840c40fc8606b3784e671cba P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.565053 28799 ts_tablet_manager.cc:1434] T 864f2602840c40fc8606b3784e671cba P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.019s user 0.001s sys 0.001s
I20260618 13:59:24.569244 28384 tablet_service.cc:1511] Processing CreateTablet for tablet c4cfca9e76e641a7bc0b58b9ee240f36 (DEFAULT_TABLE table=default.Table_0e9f574416d84a0f826fa9a1fcc56210 [id=098fa24bb71e4fe1aecc33aea1275ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.569496 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 864f2602840c40fc8606b3784e671cba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.569458 28653 tablet_service.cc:1511] Processing CreateTablet for tablet c4cfca9e76e641a7bc0b58b9ee240f36 (DEFAULT_TABLE table=default.Table_0e9f574416d84a0f826fa9a1fcc56210 [id=098fa24bb71e4fe1aecc33aea1275ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.569648 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4cfca9e76e641a7bc0b58b9ee240f36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.569831 28799 tablet_bootstrap.cc:492] T eead369de58f40378254697cbbbc6958 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:24.569867 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4cfca9e76e641a7bc0b58b9ee240f36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.570575 28798 tablet_bootstrap.cc:492] T e1a75029992645c5b942c0ec07862bb4 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:24.565452 28802 ts_tablet_manager.cc:1916] T 125bb6885ab348d885d5a3b4bdd45bc7 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.572289 28798 tablet_bootstrap.cc:654] T e1a75029992645c5b942c0ec07862bb4 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.573693 28801 ts_tablet_manager.cc:1929] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:24.573885 28801 log.cc:1199] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/311ba4dd83984f379c8f4f8274bc26c1
I20260618 13:59:24.574028 28799 tablet_bootstrap.cc:654] T eead369de58f40378254697cbbbc6958 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.574458 28801 ts_tablet_manager.cc:1950] T 311ba4dd83984f379c8f4f8274bc26c1 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:24.575491 28801 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34092 (request call id 243) took 1006 ms. Trace:
I20260618 13:59:24.575659 28801 rpcz_store.cc:276] 0618 13:59:23.569364 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:23.570239 (+ 875us) service_pool.cc:224] Handling call
0618 13:59:24.349744 (+779505us) ts_tablet_manager.cc:1151] Deleting tablet 311ba4dd83984f379c8f4f8274bc26c1
0618 13:59:24.349755 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:24.516894 (+167139us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:24.573628 (+ 56734us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:24.575430 (+ 1802us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":767580}
I20260618 13:59:24.576406 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 311ba4dd83984f379c8f4f8274bc26c1 (table default.Table_dbc49898f77e441cb501871dfae61db4 [id=d8ef984c664f4a6598c70a48e9aff826]) successfully deleted
I20260618 13:59:24.578284 28220 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet c557afc3abfa4ef188bf248e89f49d87
I20260618 13:59:24.579586 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet c557afc3abfa4ef188bf248e89f49d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.582660 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet c557afc3abfa4ef188bf248e89f49d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:24.582703 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet c557afc3abfa4ef188bf248e89f49d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.585055 28802 ts_tablet_manager.cc:1929] T 125bb6885ab348d885d5a3b4bdd45bc7 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.585642 28222 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 04dac931e62a448d99633878eb70b92f
I20260618 13:59:24.585664 28527 tablet_service.cc:1511] Processing CreateTablet for tablet c4cfca9e76e641a7bc0b58b9ee240f36 (DEFAULT_TABLE table=default.Table_0e9f574416d84a0f826fa9a1fcc56210 [id=098fa24bb71e4fe1aecc33aea1275ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.586073 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4cfca9e76e641a7bc0b58b9ee240f36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.586117 28222 catalog_manager.cc:6784] ReplaceTablet: tablet 04dac931e62a448d99633878eb70b92f deleted and replaced by tablet 15483938a06e459789f87c6bb456bd15
I20260618 13:59:24.586683 28229 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 2f3d2f236fd44ab8af0524ea282cfa94
I20260618 13:59:24.587668 28221 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 2f5a334d18ba4c06bb358fa86d8af80d
I20260618 13:59:24.587385 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 04dac931e62a448d99633878eb70b92f with delete_type TABLET_DATA_DELETED (replaced by tablet 15483938a06e459789f87c6bb456bd15) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.588536 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f3d2f236fd44ab8af0524ea282cfa94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.588598 28799 tablet_bootstrap.cc:492] T eead369de58f40378254697cbbbc6958 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:24.588768 28799 ts_tablet_manager.cc:1403] T eead369de58f40378254697cbbbc6958 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.000s
I20260618 13:59:24.588891 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f5a334d18ba4c06bb358fa86d8af80d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.589679 28799 raft_consensus.cc:359] T eead369de58f40378254697cbbbc6958 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.589787 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 04dac931e62a448d99633878eb70b92f with delete_type TABLET_DATA_DELETED (replaced by tablet 15483938a06e459789f87c6bb456bd15) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:24.589944 28799 raft_consensus.cc:385] T eead369de58f40378254697cbbbc6958 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.590070 28799 raft_consensus.cc:740] T eead369de58f40378254697cbbbc6958 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.590312 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 04dac931e62a448d99633878eb70b92f with delete_type TABLET_DATA_DELETED (replaced by tablet 15483938a06e459789f87c6bb456bd15) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.590400 28799 consensus_queue.cc:260] T eead369de58f40378254697cbbbc6958 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.590564 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f3d2f236fd44ab8af0524ea282cfa94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.591543 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f5a334d18ba4c06bb358fa86d8af80d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.592305 28648 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f5a334d18ba4c06bb358fa86d8af80d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:24.594323 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f3d2f236fd44ab8af0524ea282cfa94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:24.595024 28226 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet b51d0851efb34eaa9e9e4be6b292ef62
I20260618 13:59:24.595278 28799 ts_tablet_manager.cc:1434] T eead369de58f40378254697cbbbc6958 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260618 13:59:24.595443 28226 catalog_manager.cc:6784] ReplaceTablet: tablet b51d0851efb34eaa9e9e4be6b292ef62 deleted and replaced by tablet f9f7359c4a144ca89fe3e51d1365a425
I20260618 13:59:24.588562 28802 log.cc:1199] T 125bb6885ab348d885d5a3b4bdd45bc7 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/125bb6885ab348d885d5a3b4bdd45bc7
I20260618 13:59:24.596397 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_6dc717aafef14b22ab95830a6cf50642"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.597025 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6dc717aafef14b22ab95830a6cf50642 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.597836 28458 heartbeater.cc:344] Connected to a master server at 127.27.100.61:45967
I20260618 13:59:24.598524 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet b51d0851efb34eaa9e9e4be6b292ef62 with delete_type TABLET_DATA_DELETED (replaced by tablet f9f7359c4a144ca89fe3e51d1365a425) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:24.600250 28226 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_4de8ba099efe4bb3bc0a0d90e1861107" } new_table_name: "default.Table_f958c4ce58c34bfda98454664e202af2" modify_external_catalogs: true
I20260618 13:59:24.601013 28799 tablet_bootstrap.cc:492] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:24.602803 28799 tablet_bootstrap.cc:654] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.603926 29097 sys_catalog.cc:455] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7637285570574a6dbcef1de6f84ecac0. Latest consensus state: current_term: 1 leader_uuid: "7637285570574a6dbcef1de6f84ecac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:24.604183 29097 sys_catalog.cc:458] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 13:59:24.603156 28802 ts_tablet_manager.cc:1950] T 125bb6885ab348d885d5a3b4bdd45bc7 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:24.604441 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet b51d0851efb34eaa9e9e4be6b292ef62 with delete_type TABLET_DATA_DELETED (replaced by tablet f9f7359c4a144ca89fe3e51d1365a425) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.604665 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet eead369de58f40378254697cbbbc6958 with delete_type TABLET_DATA_DELETED (replaced by tablet d464dd402d2044469e364ed37fdfb39b) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.608970 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 125bb6885ab348d885d5a3b4bdd45bc7 (table default.Table_af38438888374cf2b0b8f3db85810efc [id=84cf9477c8584508b8e1c847cdf58b28]) successfully deleted
I20260618 13:59:24.609469 28802 tablet_replica.cc:333] T 864f2602840c40fc8606b3784e671cba P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:24.609760 28802 raft_consensus.cc:2243] T 864f2602840c40fc8606b3784e671cba P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.610008 28802 raft_consensus.cc:2272] T 864f2602840c40fc8606b3784e671cba P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.610810 28802 ts_tablet_manager.cc:1916] T 864f2602840c40fc8606b3784e671cba P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.610828 28653 tablet_service.cc:1511] Processing CreateTablet for tablet b95bdb0e36584d92b9c28de257553827 (DEFAULT_TABLE table=default.Table_259b4dd1855b42a8824f00306cb5223a [id=15c34c0db3b54b27ac29f13880f49f0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.611277 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b95bdb0e36584d92b9c28de257553827. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:24.615899 28226 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a1a06078974e4bfbbc4b949802a39bf1 consensus state has no leader
I20260618 13:59:24.616525 28221 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_1c0c9338a74e4ac8aea2fdeea2495aec" } new_table_name: "default.Table_99428babdde649b8b5d5fab801b58136" modify_external_catalogs: true
I20260618 13:59:24.620471 28385 tablet_service.cc:1511] Processing CreateTablet for tablet b95bdb0e36584d92b9c28de257553827 (DEFAULT_TABLE table=default.Table_259b4dd1855b42a8824f00306cb5223a [id=15c34c0db3b54b27ac29f13880f49f0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.620915 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b95bdb0e36584d92b9c28de257553827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.621567 28802 ts_tablet_manager.cc:1929] T 864f2602840c40fc8606b3784e671cba P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.621723 28802 log.cc:1199] T 864f2602840c40fc8606b3784e671cba P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/864f2602840c40fc8606b3784e671cba
I20260618 13:59:24.622100 28802 ts_tablet_manager.cc:1950] T 864f2602840c40fc8606b3784e671cba P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:24.629317 28799 tablet_bootstrap.cc:492] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:24.629568 28799 ts_tablet_manager.cc:1403] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.029s user 0.004s sys 0.000s
I20260618 13:59:24.629750 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet b51d0851efb34eaa9e9e4be6b292ef62 with delete_type TABLET_DATA_DELETED (replaced by tablet f9f7359c4a144ca89fe3e51d1365a425) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.630316 28801 tablet_replica.cc:333] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
W20260618 13:59:24.630343 28221 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet da4d4f03911f4ac099ecc2fa1d8af494 consensus state has no leader
I20260618 13:59:24.630540 28801 raft_consensus.cc:2243] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.631263 28801 raft_consensus.cc:2272] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.631821 28801 ts_tablet_manager.cc:1916] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.632483 28799 raft_consensus.cc:359] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.632668 28799 raft_consensus.cc:385] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.632757 28799 raft_consensus.cc:740] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.633117 28799 consensus_queue.cc:260] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.636945 28389 tablet_service.cc:1511] Processing CreateTablet for tablet a10e83d26b704201b95e7c0c44a1a4f5 (DEFAULT_TABLE table=default.Table_663ed02889aa45938d7d1316d4d19c06 [id=c79cfc8300a94a18873df52a1d83ada3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.637030 28799 ts_tablet_manager.cc:1434] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260618 13:59:24.637369 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a10e83d26b704201b95e7c0c44a1a4f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.637923 29095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7637285570574a6dbcef1de6f84ecac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:24.638671 29095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: This master's current role is: LEADER
I20260618 13:59:24.639370 28518 tablet_service.cc:1511] Processing CreateTablet for tablet b95bdb0e36584d92b9c28de257553827 (DEFAULT_TABLE table=default.Table_259b4dd1855b42a8824f00306cb5223a [id=15c34c0db3b54b27ac29f13880f49f0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.639818 28518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b95bdb0e36584d92b9c28de257553827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.640728 28799 tablet_bootstrap.cc:492] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:24.642596 28799 tablet_bootstrap.cc:654] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.644834 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 864f2602840c40fc8606b3784e671cba (table default.Table_fe8f6240599e467a92887643cc9d210b [id=3ac9a6b120d44ac2bae74d85d7bc6930]) successfully deleted
W20260618 13:59:24.645105 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet c557afc3abfa4ef188bf248e89f49d87 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c557afc3abfa4ef188bf248e89f49d87 already in progress: creating tablet
W20260618 13:59:24.645332 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 04dac931e62a448d99633878eb70b92f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 04dac931e62a448d99633878eb70b92f already in progress: creating tablet
W20260618 13:59:24.645588 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 2f3d2f236fd44ab8af0524ea282cfa94 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f3d2f236fd44ab8af0524ea282cfa94 already in progress: creating tablet
I20260618 13:59:24.646992 28522 tablet_service.cc:1511] Processing CreateTablet for tablet a10e83d26b704201b95e7c0c44a1a4f5 (DEFAULT_TABLE table=default.Table_663ed02889aa45938d7d1316d4d19c06 [id=c79cfc8300a94a18873df52a1d83ada3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.647388 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a10e83d26b704201b95e7c0c44a1a4f5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:24.650885 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 2f5a334d18ba4c06bb358fa86d8af80d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f5a334d18ba4c06bb358fa86d8af80d already in progress: creating tablet
I20260618 13:59:24.651260 28648 tablet_service.cc:1511] Processing CreateTablet for tablet 591465e2b57247a2b224cc8b09225811 (DEFAULT_TABLE table=default.Table_0a85d79af65f48a3803bfd1248593a11 [id=2f4d62be772a421ab777fd23be58b719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.651710 28648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 591465e2b57247a2b224cc8b09225811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.651723 28388 tablet_service.cc:1511] Processing CreateTablet for tablet 591465e2b57247a2b224cc8b09225811 (DEFAULT_TABLE table=default.Table_0a85d79af65f48a3803bfd1248593a11 [id=2f4d62be772a421ab777fd23be58b719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.651767 28516 tablet_service.cc:1511] Processing CreateTablet for tablet 591465e2b57247a2b224cc8b09225811 (DEFAULT_TABLE table=default.Table_0a85d79af65f48a3803bfd1248593a11 [id=2f4d62be772a421ab777fd23be58b719]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:24.651922 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet b51d0851efb34eaa9e9e4be6b292ef62 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b51d0851efb34eaa9e9e4be6b292ef62 already in progress: creating tablet
I20260618 13:59:24.652109 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 591465e2b57247a2b224cc8b09225811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.652103 28516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 591465e2b57247a2b224cc8b09225811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.652238 28802 tablet_replica.cc:333] T eead369de58f40378254697cbbbc6958 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:24.652411 28802 raft_consensus.cc:2243] T eead369de58f40378254697cbbbc6958 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.652592 28802 raft_consensus.cc:2272] T eead369de58f40378254697cbbbc6958 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.653549 28802 ts_tablet_manager.cc:1916] T eead369de58f40378254697cbbbc6958 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.661478 28799 tablet_bootstrap.cc:492] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:24.661717 28799 ts_tablet_manager.cc:1403] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.005s
I20260618 13:59:24.662523 28799 raft_consensus.cc:359] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.662734 28653 tablet_service.cc:1511] Processing CreateTablet for tablet a10e83d26b704201b95e7c0c44a1a4f5 (DEFAULT_TABLE table=default.Table_663ed02889aa45938d7d1316d4d19c06 [id=c79cfc8300a94a18873df52a1d83ada3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.662812 28799 raft_consensus.cc:385] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.662958 28799 raft_consensus.cc:740] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.663044 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet d6871232860b41d489e34d95eb897241 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.663160 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a10e83d26b704201b95e7c0c44a1a4f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.663342 28799 consensus_queue.cc:260] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.665208 28801 ts_tablet_manager.cc:1929] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.665768 28226 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet da4d4f03911f4ac099ecc2fa1d8af494 (table default.Table_99428babdde649b8b5d5fab801b58136 [id=f7cb92d4ab0742af9946f6bcf708df4d]). Expected version 1 got 0
I20260618 13:59:24.669442 28801 log.cc:1199] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/18fb8adfc8ee40e18a1da5f978317124
I20260618 13:59:24.672588 28800 ts_tablet_manager.cc:1929] T 37fedd27109944e2b3c870418c117758 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.672739 28800 log.cc:1199] T 37fedd27109944e2b3c870418c117758 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/37fedd27109944e2b3c870418c117758
I20260618 13:59:24.673206 28801 ts_tablet_manager.cc:1950] T 18fb8adfc8ee40e18a1da5f978317124 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:24.673933 28801 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34092 (request call id 245) took 1092 ms. Trace:
I20260618 13:59:24.676035 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_72e5809d8cce4f3bbe8227efcb8e2b2c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.676697 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_72e5809d8cce4f3bbe8227efcb8e2b2c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.677920 28229 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_3deacb2c9668491a85bbb09ae3f57d66" } modify_external_catalogs: true
I20260618 13:59:24.678154 28229 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_3deacb2c9668491a85bbb09ae3f57d66" } modify_external_catalogs: true
I20260618 13:59:24.679015 28799 ts_tablet_manager.cc:1434] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.017s user 0.001s sys 0.000s
I20260618 13:59:24.682729 28648 tablet_service.cc:1511] Processing CreateTablet for tablet c3d523419ff244f085d6eac1a3231973 (DEFAULT_TABLE table=default.Table_176a739c87ee45798e2470f8ec00d632 [id=2534d29e39c14366b10492a7c6a92947]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.683212 28648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3d523419ff244f085d6eac1a3231973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.683701 28799 tablet_bootstrap.cc:492] T 4a01dfb4d59549399c94640f16045555 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:24.684154 28222 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet da4d4f03911f4ac099ecc2fa1d8af494 (table default.Table_99428babdde649b8b5d5fab801b58136 [id=f7cb92d4ab0742af9946f6bcf708df4d]). Expected version 1 got 0
I20260618 13:59:24.685952 28799 tablet_bootstrap.cc:654] T 4a01dfb4d59549399c94640f16045555 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.685927 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_dfaff54a9db64b1eba1c46f23fa4382c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.686717 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dfaff54a9db64b1eba1c46f23fa4382c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.694746 28389 tablet_service.cc:1511] Processing CreateTablet for tablet c3d523419ff244f085d6eac1a3231973 (DEFAULT_TABLE table=default.Table_176a739c87ee45798e2470f8ec00d632 [id=2534d29e39c14366b10492a7c6a92947]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.695890 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3d523419ff244f085d6eac1a3231973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.696168 28229 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet ab521028ac1a4d0da013112d76bb169f
I20260618 13:59:24.697377 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet ab521028ac1a4d0da013112d76bb169f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:24.699424 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet ab521028ac1a4d0da013112d76bb169f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.699621 28229 master_service.cc:960] ReplaceTablet: received request to replace tablet 68b8293a7c924a449307ab88a1cf9290 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:24.674494 28801 rpcz_store.cc:276] 0618 13:59:23.582482 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:23.582536 (+ 54us) service_pool.cc:224] Handling call
0618 13:59:24.599337 (+1016801us) ts_tablet_manager.cc:1151] Deleting tablet 18fb8adfc8ee40e18a1da5f978317124
0618 13:59:24.599352 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:24.635390 (+ 36038us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:24.665123 (+ 29733us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:24.673899 (+ 8776us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1002442}
I20260618 13:59:24.701894 28934 raft_consensus.cc:493] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:24.702059 28934 raft_consensus.cc:515] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.702741 28660 tablet_service.cc:1511] Processing CreateTablet for tablet b0d3d363766546c4a97b0763bad11f71 (DEFAULT_TABLE table=default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.702812 28934 leader_election.cc:290] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:24.703172 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0d3d363766546c4a97b0763bad11f71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.703248 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6871232860b41d489e34d95eb897241" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:24.703783 28346 leader_election.cc:343] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:24.704533 28800 ts_tablet_manager.cc:1950] T 37fedd27109944e2b3c870418c117758 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:24.705515 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 37fedd27109944e2b3c870418c117758 (table default.Table_e9f01ff23601409189a5923e3755f4f8 [id=97b635f1072a4d82a82753d618600d4a]) successfully deleted
W20260618 13:59:24.706104 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet c557afc3abfa4ef188bf248e89f49d87 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c557afc3abfa4ef188bf248e89f49d87 already in progress: creating tablet
W20260618 13:59:24.706686 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 04dac931e62a448d99633878eb70b92f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 04dac931e62a448d99633878eb70b92f already in progress: creating tablet
W20260618 13:59:24.707479 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 2f5a334d18ba4c06bb358fa86d8af80d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f5a334d18ba4c06bb358fa86d8af80d already in progress: creating tablet
I20260618 13:59:24.709674 28385 tablet_service.cc:1511] Processing CreateTablet for tablet b0d3d363766546c4a97b0763bad11f71 (DEFAULT_TABLE table=default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.709702 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_7910db46cf394cdebe608e7efb3388cc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:24.710064 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0d3d363766546c4a97b0763bad11f71. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:24.710177 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7910db46cf394cdebe608e7efb3388cc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.712440 28799 tablet_bootstrap.cc:492] T 4a01dfb4d59549399c94640f16045555 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:24.712652 28799 ts_tablet_manager.cc:1403] T 4a01dfb4d59549399c94640f16045555 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.029s user 0.008s sys 0.000s
I20260618 13:59:24.714133 28799 raft_consensus.cc:359] T 4a01dfb4d59549399c94640f16045555 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.714390 28799 raft_consensus.cc:385] T 4a01dfb4d59549399c94640f16045555 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.714557 28799 raft_consensus.cc:740] T 4a01dfb4d59549399c94640f16045555 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
W20260618 13:59:24.714916 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 2f3d2f236fd44ab8af0524ea282cfa94 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f3d2f236fd44ab8af0524ea282cfa94 already in progress: creating tablet
I20260618 13:59:24.714892 28799 consensus_queue.cc:260] T 4a01dfb4d59549399c94640f16045555 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.715552 28222 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_9d7999641cc344f18ccc672c49518192" } modify_external_catalogs: true
I20260618 13:59:24.715730 28222 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_9d7999641cc344f18ccc672c49518192" } modify_external_catalogs: true
I20260618 13:59:24.717346 28229 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_e24e443c392040b0adf1e8c8eb581a10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:24.717633 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_3ef2ff183a6044e2a9e069a4eac133b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.718137 28229 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e24e443c392040b0adf1e8c8eb581a10 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:24.718546 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3ef2ff183a6044e2a9e069a4eac133b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.719358 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_4108c7ea79f5426395746346c44df521"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:24.719501 28799 ts_tablet_manager.cc:1434] T 4a01dfb4d59549399c94640f16045555 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
W20260618 13:59:24.719826 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4108c7ea79f5426395746346c44df521 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.723001 28799 tablet_bootstrap.cc:492] T edc72a06ee8e4420b07a17d2feb3fcc2 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:24.724823 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet 4a01dfb4d59549399c94640f16045555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
W20260618 13:59:24.726967 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet b51d0851efb34eaa9e9e4be6b292ef62 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b51d0851efb34eaa9e9e4be6b292ef62 already in progress: creating tablet
I20260618 13:59:24.727591 28799 tablet_bootstrap.cc:654] T edc72a06ee8e4420b07a17d2feb3fcc2 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.711830 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6871232860b41d489e34d95eb897241" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:24.728176 28346 leader_election.cc:343] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:24.728334 28346 leader_election.cc:304] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:24.728596 28934 raft_consensus.cc:2749] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:24.714721 28516 tablet_service.cc:1511] Processing CreateTablet for tablet b0d3d363766546c4a97b0763bad11f71 (DEFAULT_TABLE table=default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.699318 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet ab521028ac1a4d0da013112d76bb169f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.720297 28527 tablet_service.cc:1511] Processing CreateTablet for tablet c3d523419ff244f085d6eac1a3231973 (DEFAULT_TABLE table=default.Table_176a739c87ee45798e2470f8ec00d632 [id=2534d29e39c14366b10492a7c6a92947]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.729125 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 3a0039a463ec4a9c82a21a3a69b565fc (DEFAULT_TABLE table=default.Table_8c1fdd54393c4ad383e72a07837ead2a [id=6a7e8e3becb64fd5bc583ee08eaa4437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.729516 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a0039a463ec4a9c82a21a3a69b565fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.731559 28222 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 5c75278db2234b11bf95f1cf5cc76edd
I20260618 13:59:24.732611 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c75278db2234b11bf95f1cf5cc76edd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.733566 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c75278db2234b11bf95f1cf5cc76edd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:24.733767 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 18fb8adfc8ee40e18a1da5f978317124 (table default.Table_bf7cbaec816a4c04a225ff31f57123ed [id=f34f77924d464341afa518830ba7bbbe]) successfully deleted
W20260618 13:59:24.734007 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 8b59255b14df4158afe37cb8890e7415 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b59255b14df4158afe37cb8890e7415 already in progress: creating tablet
I20260618 13:59:24.734112 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c75278db2234b11bf95f1cf5cc76edd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:24.734242 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet e1a75029992645c5b942c0ec07862bb4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e1a75029992645c5b942c0ec07862bb4 already in progress: creating tablet
W20260618 13:59:24.736075 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet c84189a4b1194c87bbdf81f6e0ed56e4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c84189a4b1194c87bbdf81f6e0ed56e4 already in progress: creating tablet
I20260618 13:59:24.737377 28798 tablet_bootstrap.cc:492] T e1a75029992645c5b942c0ec07862bb4 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:24.738371 28798 ts_tablet_manager.cc:1403] T e1a75029992645c5b942c0ec07862bb4 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.168s user 0.023s sys 0.000s
I20260618 13:59:24.738889 28222 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_6598b23b0b264b08b63faa03d23e87a9" } new_table_name: "default.Table_1af4cab3531c47b5a9cd4fee3f8251c2" modify_external_catalogs: true
I20260618 13:59:24.737711 28801 tablet_replica.cc:333] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:24.738950 28220 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_de7bace9efe6428da6ea8e662f45c90e" } new_table_name: "default.Table_031e1a03030b45c99c1270d612008f91" modify_external_catalogs: true
I20260618 13:59:24.739607 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_122b5d1531834ee2aa6bbc6593efb740"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.740120 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_122b5d1531834ee2aa6bbc6593efb740 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:24.740581 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 3752f49c9fc64071a45e0ce4ff4cfef2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3752f49c9fc64071a45e0ce4ff4cfef2 already in progress: creating tablet
I20260618 13:59:24.741072 28229 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_dfaff54a9db64b1eba1c46f23fa4382c" } new_table_name: "default.Table_55f1c253e54246eda98a7dcf2996952b" modify_external_catalogs: true
I20260618 13:59:24.741247 28225 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_fd6ef45aeec84b48b8fb7a54db4b4f81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.741626 28215 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet ab521028ac1a4d0da013112d76bb169f because the tablet was not found. No further retry: Not found: Tablet not found: ab521028ac1a4d0da013112d76bb169f
W20260618 13:59:24.741765 28225 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd6ef45aeec84b48b8fb7a54db4b4f81 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:24.741878 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 5c75278db2234b11bf95f1cf5cc76edd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5c75278db2234b11bf95f1cf5cc76edd already in progress: creating tablet
I20260618 13:59:24.743649 28801 raft_consensus.cc:2243] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.743885 28801 raft_consensus.cc:2272] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.743876 28384 tablet_service.cc:1511] Processing CreateTablet for tablet 3a0039a463ec4a9c82a21a3a69b565fc (DEFAULT_TABLE table=default.Table_8c1fdd54393c4ad383e72a07837ead2a [id=6a7e8e3becb64fd5bc583ee08eaa4437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.744370 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a0039a463ec4a9c82a21a3a69b565fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.744577 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_9cf9f72ae27b44bc9aa43303a7103214"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:24.745195 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9cf9f72ae27b44bc9aa43303a7103214 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.745620 28799 tablet_bootstrap.cc:492] T edc72a06ee8e4420b07a17d2feb3fcc2 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:24.745849 28799 ts_tablet_manager.cc:1403] T edc72a06ee8e4420b07a17d2feb3fcc2 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.004s
I20260618 13:59:24.746909 28799 raft_consensus.cc:359] T edc72a06ee8e4420b07a17d2feb3fcc2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.747171 28799 raft_consensus.cc:385] T edc72a06ee8e4420b07a17d2feb3fcc2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.747308 28799 raft_consensus.cc:740] T edc72a06ee8e4420b07a17d2feb3fcc2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.747671 28799 consensus_queue.cc:260] T edc72a06ee8e4420b07a17d2feb3fcc2 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.752694 28799 ts_tablet_manager.cc:1434] T edc72a06ee8e4420b07a17d2feb3fcc2 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260618 13:59:24.755728 28799 tablet_bootstrap.cc:492] T 94365c44b5e24acab5df60c67203794a P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:24.755887 28798 raft_consensus.cc:359] T e1a75029992645c5b942c0ec07862bb4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.756079 28798 raft_consensus.cc:385] T e1a75029992645c5b942c0ec07862bb4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.756175 28798 raft_consensus.cc:740] T e1a75029992645c5b942c0ec07862bb4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.757818 28799 tablet_bootstrap.cc:654] T 94365c44b5e24acab5df60c67203794a P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.762814 28801 ts_tablet_manager.cc:1916] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.695152 28802 ts_tablet_manager.cc:1929] T eead369de58f40378254697cbbbc6958 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.765872 28802 log.cc:1199] T eead369de58f40378254697cbbbc6958 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/eead369de58f40378254697cbbbc6958
I20260618 13:59:24.767181 28802 ts_tablet_manager.cc:1950] T eead369de58f40378254697cbbbc6958 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:24.769202 28798 consensus_queue.cc:260] T e1a75029992645c5b942c0ec07862bb4 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.771481 28518 tablet_service.cc:1511] Processing CreateTablet for tablet 3a0039a463ec4a9c82a21a3a69b565fc (DEFAULT_TABLE table=default.Table_8c1fdd54393c4ad383e72a07837ead2a [id=6a7e8e3becb64fd5bc583ee08eaa4437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.777145 28227 master_service.cc:960] ReplaceTablet: received request to replace tablet d502d67826ff4d7a8b0f515e5d8b6d77 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:24.777827 28516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0d3d363766546c4a97b0763bad11f71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.782563 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3d523419ff244f085d6eac1a3231973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.782881 28518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a0039a463ec4a9c82a21a3a69b565fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.788607 28648 tablet_service.cc:1511] Processing CreateTablet for tablet 0463fa2e30554abcbfc8f37b702e81d4 (DEFAULT_TABLE table=default.Table_ff3779f2af2443428f9c1e3772091659 [id=c3a54561a2e7448ebb47b0234579944e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.789026 28648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0463fa2e30554abcbfc8f37b702e81d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.795702 28224 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_dac9c04f74664cce9c065edc3e491bf6" } modify_external_catalogs: true
I20260618 13:59:24.795778 28223 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_3cdafadde7c146d38bdb87c358269c35" } modify_external_catalogs: true
I20260618 13:59:24.795960 28223 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_3cdafadde7c146d38bdb87c358269c35" } modify_external_catalogs: true
I20260618 13:59:24.795939 28224 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_dac9c04f74664cce9c065edc3e491bf6" } modify_external_catalogs: true
W20260618 13:59:24.807144 28220 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9b75ece59d924e3db49e002bf0faa89c consensus state has no leader
I20260618 13:59:24.816066 28798 ts_tablet_manager.cc:1434] T e1a75029992645c5b942c0ec07862bb4 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.077s user 0.002s sys 0.000s
I20260618 13:59:24.767395 28388 tablet_service.cc:1558] Processing DeleteTablet for tablet edc72a06ee8e4420b07a17d2feb3fcc2 with delete_type TABLET_DATA_DELETED (replaced by tablet aa8cf927bed9437086efd8e0f21ba26a) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.768728 28799 tablet_bootstrap.cc:492] T 94365c44b5e24acab5df60c67203794a P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:24.819067 28799 ts_tablet_manager.cc:1403] T 94365c44b5e24acab5df60c67203794a P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.063s user 0.004s sys 0.002s
I20260618 13:59:24.819897 28799 raft_consensus.cc:359] T 94365c44b5e24acab5df60c67203794a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.820083 28799 raft_consensus.cc:385] T 94365c44b5e24acab5df60c67203794a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.820047 28220 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_f75651a1e55f4271b39388713dcd74aa" } modify_external_catalogs: true
I20260618 13:59:24.820176 28799 raft_consensus.cc:740] T 94365c44b5e24acab5df60c67203794a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.820298 28220 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_f75651a1e55f4271b39388713dcd74aa" } modify_external_catalogs: true
I20260618 13:59:24.820410 28799 consensus_queue.cc:260] T 94365c44b5e24acab5df60c67203794a P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.820917 28799 ts_tablet_manager.cc:1434] T 94365c44b5e24acab5df60c67203794a P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:24.812521 28391 tablet_service.cc:1511] Processing CreateTablet for tablet 0463fa2e30554abcbfc8f37b702e81d4 (DEFAULT_TABLE table=default.Table_ff3779f2af2443428f9c1e3772091659 [id=c3a54561a2e7448ebb47b0234579944e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.821473 28799 tablet_bootstrap.cc:492] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:24.823305 28799 tablet_bootstrap.cc:654] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.824504 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0463fa2e30554abcbfc8f37b702e81d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.826155 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet e1a75029992645c5b942c0ec07862bb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:24.826320 28377 tablet_service.cc:1511] Processing CreateTablet for tablet 015494374645469785f1f978eaf0e9a5 (DEFAULT_TABLE table=default.Table_7699960fdee24242bf51bcfcbd6f68c2 [id=3b9ab803536344d3a59ff77b2b58ae31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.826718 28377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 015494374645469785f1f978eaf0e9a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.829600 28653 tablet_service.cc:1511] Processing CreateTablet for tablet 015494374645469785f1f978eaf0e9a5 (DEFAULT_TABLE table=default.Table_7699960fdee24242bf51bcfcbd6f68c2 [id=3b9ab803536344d3a59ff77b2b58ae31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.829617 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_0d8a6f9e3b84409e98be6ee8668907e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:24.829974 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 015494374645469785f1f978eaf0e9a5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:24.830107 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0d8a6f9e3b84409e98be6ee8668907e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:24.831102 28798 tablet_bootstrap.cc:492] T b1f734a3f29b42e591d5564e99292b5e P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:24.833336 28798 tablet_bootstrap.cc:654] T b1f734a3f29b42e591d5564e99292b5e P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.834770 28226 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_d4895ce143324c7f9c575472ce329ed3" } new_table_name: "default.Table_6454f30d08d14030a0a405bcad4e06b3" modify_external_catalogs: true
I20260618 13:59:24.836027 28810 raft_consensus.cc:493] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:24.836179 28810 raft_consensus.cc:515] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.836779 28810 leader_election.cc:290] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:24.837746 28483 leader_election.cc:343] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 37fedd27109944e2b3c870418c117758
W20260618 13:59:24.837999 28484 leader_election.cc:343] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 37fedd27109944e2b3c870418c117758
I20260618 13:59:24.838126 28484 leader_election.cc:304] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:24.838486 28810 raft_consensus.cc:2749] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:24.842681 28388 tablet_service.cc:1558] Processing DeleteTablet for tablet 94365c44b5e24acab5df60c67203794a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.846239 28799 tablet_bootstrap.cc:492] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:24.846411 28799 ts_tablet_manager.cc:1403] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.001s
I20260618 13:59:24.847259 28799 raft_consensus.cc:359] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.847472 28799 raft_consensus.cc:385] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.847561 28799 raft_consensus.cc:740] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.847821 28799 consensus_queue.cc:260] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.848881 28797 tablet_bootstrap.cc:492] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:24.849802 28799 ts_tablet_manager.cc:1434] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:24.850100 28799 tablet_bootstrap.cc:492] T 8b59255b14df4158afe37cb8890e7415 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:24.850654 28458 heartbeater.cc:344] Connected to a master server at 127.27.100.61:45967
I20260618 13:59:24.851202 28801 ts_tablet_manager.cc:1929] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.851343 28801 log.cc:1199] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/4ebed81a21b14db48ecb69299e0956cd
I20260618 13:59:24.851671 28801 ts_tablet_manager.cc:1950] T 4ebed81a21b14db48ecb69299e0956cd P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:24.851848 28799 tablet_bootstrap.cc:654] T 8b59255b14df4158afe37cb8890e7415 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.854005 28798 tablet_bootstrap.cc:492] T b1f734a3f29b42e591d5564e99292b5e P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:24.854174 28798 ts_tablet_manager.cc:1403] T b1f734a3f29b42e591d5564e99292b5e P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.000s
I20260618 13:59:24.854406 29026 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" instance_seqno: 1781791158100017) as {username='slave'} at 127.27.100.1:48495; Asking this server to re-register.
I20260618 13:59:24.854971 28798 raft_consensus.cc:359] T b1f734a3f29b42e591d5564e99292b5e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.855165 28798 raft_consensus.cc:385] T b1f734a3f29b42e591d5564e99292b5e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.855237 28798 raft_consensus.cc:740] T b1f734a3f29b42e591d5564e99292b5e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.855788 28458 heartbeater.cc:461] Registering TS with master...
I20260618 13:59:24.858191 29026 ts_manager.cc:194] Registered new tserver with Master: 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:24.862594 28798 consensus_queue.cc:260] T b1f734a3f29b42e591d5564e99292b5e P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.870671 29026 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "97859748af2c41879ba2a668dda999d2" instance_seqno: 1781791158554469) as {username='slave'} at 127.27.100.2:52853; Asking this server to re-register.
I20260618 13:59:24.870764 28799 tablet_bootstrap.cc:492] T 8b59255b14df4158afe37cb8890e7415 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:24.871346 28799 ts_tablet_manager.cc:1403] T 8b59255b14df4158afe37cb8890e7415 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.004s
I20260618 13:59:24.871665 28798 ts_tablet_manager.cc:1434] T b1f734a3f29b42e591d5564e99292b5e P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.017s user 0.002s sys 0.000s
I20260618 13:59:24.872056 28801 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34092 (request call id 266) took 1006 ms. Trace:
I20260618 13:59:24.872193 28799 raft_consensus.cc:359] T 8b59255b14df4158afe37cb8890e7415 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.872359 28799 raft_consensus.cc:385] T 8b59255b14df4158afe37cb8890e7415 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.872462 28799 raft_consensus.cc:740] T 8b59255b14df4158afe37cb8890e7415 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.872700 28799 consensus_queue.cc:260] T 8b59255b14df4158afe37cb8890e7415 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.872428 28596 heartbeater.cc:461] Registering TS with master...
I20260618 13:59:24.874352 28801 rpcz_store.cc:276] 0618 13:59:23.865817 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:23.865921 (+ 104us) service_pool.cc:224] Handling call
0618 13:59:24.871876 (+1005955us) ts_tablet_manager.cc:1151] Deleting tablet 0fae928833e44d2ba860d6184d4c0142
0618 13:59:24.871884 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:24.871984 (+ 100us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":214,"tablet-delete.queue_time_us":1005696}
I20260618 13:59:24.874984 28799 ts_tablet_manager.cc:1434] T 8b59255b14df4158afe37cb8890e7415 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:24.874809 28798 tablet_bootstrap.cc:492] T b739fa1847184290833e13a2515f3a3a P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:24.875718 28799 tablet_bootstrap.cc:492] T d9c00003300246ada2a356e3e1cf684b P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:24.876678 29025 ts_manager.cc:194] Registered new tserver with Master: 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:24.877435 28799 tablet_bootstrap.cc:654] T d9c00003300246ada2a356e3e1cf684b P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.862892 28797 ts_tablet_manager.cc:1403] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.311s user 0.002s sys 0.002s
I20260618 13:59:24.880540 29133 catalog_manager.cc:1294] Loaded cluster ID: 7094aaeda1bc41d1ac58a313dfa3bf98
I20260618 13:59:24.880755 28798 tablet_bootstrap.cc:654] T b739fa1847184290833e13a2515f3a3a P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.881659 28797 raft_consensus.cc:359] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.881910 28797 raft_consensus.cc:385] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.881987 28797 raft_consensus.cc:740] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.880697 29133 catalog_manager.cc:1587] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: loading cluster ID for follower catalog manager: success
I20260618 13:59:24.882330 28797 consensus_queue.cc:260] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.882500 28801 tablet_replica.cc:333] T ebf502f25e9b4342bfaa58f42db7f2f5 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:24.882689 28801 raft_consensus.cc:2243] T ebf502f25e9b4342bfaa58f42db7f2f5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.883281 28797 ts_tablet_manager.cc:1434] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:24.883750 28797 tablet_bootstrap.cc:492] T ffd7d5ebaa94416a8eaf3e61bed945e1 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:24.884402 29025 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" instance_seqno: 1781791159034721) as {username='slave'} at 127.27.100.3:42065; Asking this server to re-register.
I20260618 13:59:24.885134 28730 heartbeater.cc:461] Registering TS with master...
I20260618 13:59:24.886574 29025 ts_manager.cc:194] Registered new tserver with Master: d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:24.882946 28801 raft_consensus.cc:2272] T ebf502f25e9b4342bfaa58f42db7f2f5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.891323 28801 ts_tablet_manager.cc:1916] T ebf502f25e9b4342bfaa58f42db7f2f5 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.892640 29133 catalog_manager.cc:1609] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: acquiring CA information for follower catalog manager: success
I20260618 13:59:24.896262 28518 tablet_service.cc:1511] Processing CreateTablet for tablet 0463fa2e30554abcbfc8f37b702e81d4 (DEFAULT_TABLE table=default.Table_ff3779f2af2443428f9c1e3772091659 [id=c3a54561a2e7448ebb47b0234579944e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.896842 28518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0463fa2e30554abcbfc8f37b702e81d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.897500 29133 catalog_manager.cc:1637] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 13:59:24.903337 28798 tablet_bootstrap.cc:492] T b739fa1847184290833e13a2515f3a3a P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:24.903523 28798 ts_tablet_manager.cc:1403] T b739fa1847184290833e13a2515f3a3a P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.000s
I20260618 13:59:24.904372 28798 raft_consensus.cc:359] T b739fa1847184290833e13a2515f3a3a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.904585 28798 raft_consensus.cc:385] T b739fa1847184290833e13a2515f3a3a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.904717 28798 raft_consensus.cc:740] T b739fa1847184290833e13a2515f3a3a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.905004 28798 consensus_queue.cc:260] T b739fa1847184290833e13a2515f3a3a P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.907646 28802 tablet_replica.cc:333] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:24.907941 28802 raft_consensus.cc:2243] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.908138 28802 raft_consensus.cc:2272] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.908219 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet eead369de58f40378254697cbbbc6958 (table default.Table_e8ae99dfd8e84d7bb2723eb5fe129c2f [id=6d8f3597814241f3bc8c920d3a59008e]) successfully deleted
I20260618 13:59:24.908937 28802 ts_tablet_manager.cc:1916] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.909466 28798 ts_tablet_manager.cc:1434] T b739fa1847184290833e13a2515f3a3a P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260618 13:59:24.912508 28798 tablet_bootstrap.cc:492] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:24.916725 28798 tablet_bootstrap.cc:654] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.917096 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 4ebed81a21b14db48ecb69299e0956cd (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]) successfully deleted
W20260618 13:59:24.917915 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 0fae928833e44d2ba860d6184d4c0142 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0fae928833e44d2ba860d6184d4c0142 already in progress: creating tablet
W20260618 13:59:24.918136 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet bc0adedecee4405e86b0a37b9e62e352 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc0adedecee4405e86b0a37b9e62e352 already in progress: creating tablet
W20260618 13:59:24.918382 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet f89c9954d48e407c96c294c26644e25c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f89c9954d48e407c96c294c26644e25c already in progress: creating tablet
W20260618 13:59:24.918576 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 68b8293a7c924a449307ab88a1cf9290 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68b8293a7c924a449307ab88a1cf9290 already in progress: creating tablet
W20260618 13:59:24.918766 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 08a2dee3ac5a4bec96445a6001717eda because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 08a2dee3ac5a4bec96445a6001717eda already in progress: creating tablet
W20260618 13:59:24.918934 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet b0c9d4131ea24809ac081e7a73a21e1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0c9d4131ea24809ac081e7a73a21e1c already in progress: creating tablet
I20260618 13:59:24.920908 28802 ts_tablet_manager.cc:1929] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.921432 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 2df9e9e811ec4c2ea68467b1294bb62e (DEFAULT_TABLE table=default.Table_caa35bd6fefa4c5bab90b4b7f1ec48d9 [id=a508a0cb72474a748aacdfd918e3782a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.921811 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2df9e9e811ec4c2ea68467b1294bb62e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.925769 28377 tablet_service.cc:1511] Processing CreateTablet for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (DEFAULT_TABLE table=default.Table_6ba7e86569384c888e4208d3414512ae [id=cf981768efab4f4c9cc5353ac9c053a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:24.926136 28377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09cbe2337dd14da1b8e0629ded7e7ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:24.928148 28801 ts_tablet_manager.cc:1929] T ebf502f25e9b4342bfaa58f42db7f2f5 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.928294 28801 log.cc:1199] T ebf502f25e9b4342bfaa58f42db7f2f5 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/ebf502f25e9b4342bfaa58f42db7f2f5
I20260618 13:59:24.928702 28801 ts_tablet_manager.cc:1950] T ebf502f25e9b4342bfaa58f42db7f2f5 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:24.921121 28802 log.cc:1199] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/d6871232860b41d489e34d95eb897241
I20260618 13:59:24.934530 28801 tablet_replica.cc:333] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:24.934722 28802 ts_tablet_manager.cc:1950] T d6871232860b41d489e34d95eb897241 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:24.934813 28801 raft_consensus.cc:2243] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.935017 28801 raft_consensus.cc:2272] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.935544 28801 ts_tablet_manager.cc:1916] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.935993 28802 tablet_replica.cc:333] T 4a01dfb4d59549399c94640f16045555 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:24.936221 28802 raft_consensus.cc:2243] T 4a01dfb4d59549399c94640f16045555 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.936463 28802 raft_consensus.cc:2272] T 4a01dfb4d59549399c94640f16045555 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.936661 28225 master_service.cc:960] ReplaceTablet: received request to replace tablet e95c88e4b0a046a3ad69ee49af47c7c3 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:24.937585 28802 ts_tablet_manager.cc:1916] T 4a01dfb4d59549399c94640f16045555 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.937908 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet d6871232860b41d489e34d95eb897241 (table default.Table_c18d75e5991b440caf136e9248a572da [id=5a3cb5befb614a70b4ecf8ccfedf8ecd]) successfully deleted
W20260618 13:59:24.938181 28214 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet ab521028ac1a4d0da013112d76bb169f because the tablet was not found. No further retry: Not found: Tablet not found: ab521028ac1a4d0da013112d76bb169f
I20260618 13:59:24.937207 28225 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_37a0a01f533341e78854144439ab6dc3" } new_table_name: "default.Table_9bfb035b965742abaaf21413be078bf6" modify_external_catalogs: true
I20260618 13:59:24.941854 28799 tablet_bootstrap.cc:492] T d9c00003300246ada2a356e3e1cf684b P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:24.942060 28799 ts_tablet_manager.cc:1403] T d9c00003300246ada2a356e3e1cf684b P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.066s user 0.014s sys 0.010s
I20260618 13:59:24.942904 28799 raft_consensus.cc:359] T d9c00003300246ada2a356e3e1cf684b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.943152 28799 raft_consensus.cc:385] T d9c00003300246ada2a356e3e1cf684b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.943272 28799 raft_consensus.cc:740] T d9c00003300246ada2a356e3e1cf684b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.943542 28799 consensus_queue.cc:260] T d9c00003300246ada2a356e3e1cf684b P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:24.947460 28229 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9024952ab8094fcda1329562bbde8ef5 has no consensus state
I20260618 13:59:24.947539 28799 ts_tablet_manager.cc:1434] T d9c00003300246ada2a356e3e1cf684b P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260618 13:59:24.950578 28799 tablet_bootstrap.cc:492] T 36b8a46a41e543c58a946b6764618c7d P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:24.951695 28801 ts_tablet_manager.cc:1929] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.951869 28801 log.cc:1199] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/37fedd27109944e2b3c870418c117758
I20260618 13:59:24.952167 28229 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet a1a06078974e4bfbbc4b949802a39bf1 (table default.Table_f958c4ce58c34bfda98454664e202af2 [id=7b747ef3b2494ca78c499c5bb75648d4]). Expected version 2 got 0
I20260618 13:59:24.952301 28801 ts_tablet_manager.cc:1950] T 37fedd27109944e2b3c870418c117758 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:24.952340 28799 tablet_bootstrap.cc:654] T 36b8a46a41e543c58a946b6764618c7d P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.953112 28229 master_service.cc:960] ReplaceTablet: received request to replace tablet 2f5a334d18ba4c06bb358fa86d8af80d from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:24.954102 28229 master_service.cc:960] ReplaceTablet: received request to replace tablet ba0d05ea44e1473fb4b24f61eab096e4 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:24.955209 28229 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_28c746533f45405791fc12912e8fbcd4" } modify_external_catalogs: true
I20260618 13:59:24.955852 28377 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b59255b14df4158afe37cb8890e7415 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.956558 28802 ts_tablet_manager.cc:1929] T 4a01dfb4d59549399c94640f16045555 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.956737 28802 log.cc:1199] T 4a01dfb4d59549399c94640f16045555 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/4a01dfb4d59549399c94640f16045555
I20260618 13:59:24.956977 28798 tablet_bootstrap.cc:492] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:24.957134 28798 ts_tablet_manager.cc:1403] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.045s user 0.005s sys 0.013s
I20260618 13:59:24.957278 28229 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_28c746533f45405791fc12912e8fbcd4" } modify_external_catalogs: true
I20260618 13:59:24.957451 28802 ts_tablet_manager.cc:1950] T 4a01dfb4d59549399c94640f16045555 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:24.957881 28798 raft_consensus.cc:359] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.958087 28798 raft_consensus.cc:385] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.958171 28798 raft_consensus.cc:740] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.958482 28798 consensus_queue.cc:260] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.959386 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 4a01dfb4d59549399c94640f16045555 (table default.Table_7700ee6bae434a01b621ad02088cc5f0 [id=fe7eabed8da6400bbb721520c6182f8c]) successfully deleted
W20260618 13:59:24.960321 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 5c75278db2234b11bf95f1cf5cc76edd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5c75278db2234b11bf95f1cf5cc76edd already in progress: creating tablet
I20260618 13:59:24.960600 28802 tablet_replica.cc:333] T edc72a06ee8e4420b07a17d2feb3fcc2 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:24.960811 28802 raft_consensus.cc:2243] T edc72a06ee8e4420b07a17d2feb3fcc2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.961020 28802 raft_consensus.cc:2272] T edc72a06ee8e4420b07a17d2feb3fcc2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.961797 28801 tablet_replica.cc:333] T d980cda1c60e4edca7b4d7537f4f5837 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:24.962064 28801 raft_consensus.cc:2243] T d980cda1c60e4edca7b4d7537f4f5837 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.962208 28799 tablet_bootstrap.cc:492] T 36b8a46a41e543c58a946b6764618c7d P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:24.962332 28801 raft_consensus.cc:2272] T d980cda1c60e4edca7b4d7537f4f5837 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.962344 28799 ts_tablet_manager.cc:1403] T 36b8a46a41e543c58a946b6764618c7d P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.002s
I20260618 13:59:24.962895 28801 ts_tablet_manager.cc:1916] T d980cda1c60e4edca7b4d7537f4f5837 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.963140 28799 raft_consensus.cc:359] T 36b8a46a41e543c58a946b6764618c7d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.962090 28798 ts_tablet_manager.cc:1434] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260618 13:59:24.963789 28799 raft_consensus.cc:385] T 36b8a46a41e543c58a946b6764618c7d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.963888 28799 raft_consensus.cc:740] T 36b8a46a41e543c58a946b6764618c7d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.964135 28799 consensus_queue.cc:260] T 36b8a46a41e543c58a946b6764618c7d P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.966542 28799 ts_tablet_manager.cc:1434] T 36b8a46a41e543c58a946b6764618c7d P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260618 13:59:24.966856 28799 tablet_bootstrap.cc:492] T c84189a4b1194c87bbdf81f6e0ed56e4 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:24.968720 28802 ts_tablet_manager.cc:1916] T edc72a06ee8e4420b07a17d2feb3fcc2 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.971190 28798 tablet_bootstrap.cc:492] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:24.970247 28799 tablet_bootstrap.cc:654] T c84189a4b1194c87bbdf81f6e0ed56e4 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:24.972775 28222 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9c17e76ae9b44207998a43e3576908c4 consensus state has no leader
I20260618 13:59:24.972898 28798 tablet_bootstrap.cc:654] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.973001 28801 ts_tablet_manager.cc:1929] T d980cda1c60e4edca7b4d7537f4f5837 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.976735 28377 tablet_service.cc:1558] Processing DeleteTablet for tablet d9c00003300246ada2a356e3e1cf684b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.979112 28802 ts_tablet_manager.cc:1929] T edc72a06ee8e4420b07a17d2feb3fcc2 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:24.979569 28802 log.cc:1199] T edc72a06ee8e4420b07a17d2feb3fcc2 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/edc72a06ee8e4420b07a17d2feb3fcc2
I20260618 13:59:24.980154 28802 ts_tablet_manager.cc:1950] T edc72a06ee8e4420b07a17d2feb3fcc2 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:24.981547 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet edc72a06ee8e4420b07a17d2feb3fcc2 (table default.Table_cf80fd5bbdde4cfb855c8fbe860a15f3 [id=9d63ad7327a44124af817b465a9ae0da]) successfully deleted
I20260618 13:59:24.982525 28801 log.cc:1199] T d980cda1c60e4edca7b4d7537f4f5837 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/d980cda1c60e4edca7b4d7537f4f5837
I20260618 13:59:24.982685 28802 tablet_replica.cc:333] T 94365c44b5e24acab5df60c67203794a P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:24.982955 28801 ts_tablet_manager.cc:1950] T d980cda1c60e4edca7b4d7537f4f5837 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:24.983150 28802 raft_consensus.cc:2243] T 94365c44b5e24acab5df60c67203794a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.983645 28802 raft_consensus.cc:2272] T 94365c44b5e24acab5df60c67203794a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.983958 28801 tablet_replica.cc:333] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:24.984232 28801 raft_consensus.cc:2243] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:24.984440 28801 raft_consensus.cc:2272] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:24.984464 28227 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet d502d67826ff4d7a8b0f515e5d8b6d77
I20260618 13:59:24.984748 28802 ts_tablet_manager.cc:1916] T 94365c44b5e24acab5df60c67203794a P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.984943 28227 catalog_manager.cc:6784] ReplaceTablet: tablet d502d67826ff4d7a8b0f515e5d8b6d77 deleted and replaced by tablet 657e55fa53c447babfe91b1582806e6b
I20260618 13:59:24.984983 28801 ts_tablet_manager.cc:1916] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:24.985021 28223 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet bce7bf74b6d24d0bb43808b8a62cf1b7
I20260618 13:59:24.985317 28224 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet d1baf86497d442dbbbddbc3498bebbee
I20260618 13:59:24.985348 28377 tablet_service.cc:1558] Processing DeleteTablet for tablet d502d67826ff4d7a8b0f515e5d8b6d77 with delete_type TABLET_DATA_DELETED (replaced by tablet 657e55fa53c447babfe91b1582806e6b) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.985937 28377 tablet_service.cc:1558] Processing DeleteTablet for tablet d1baf86497d442dbbbddbc3498bebbee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.986734 28377 tablet_service.cc:1558] Processing DeleteTablet for tablet 36b8a46a41e543c58a946b6764618c7d with delete_type TABLET_DATA_DELETED (replaced by tablet f89c9954d48e407c96c294c26644e25c) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.987103 28799 tablet_bootstrap.cc:492] T c84189a4b1194c87bbdf81f6e0ed56e4 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:24.987406 28377 tablet_service.cc:1558] Processing DeleteTablet for tablet bce7bf74b6d24d0bb43808b8a62cf1b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.987672 28799 ts_tablet_manager.cc:1403] T c84189a4b1194c87bbdf81f6e0ed56e4 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.004s
I20260618 13:59:24.988863 28798 tablet_bootstrap.cc:492] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:24.988827 28799 raft_consensus.cc:359] T c84189a4b1194c87bbdf81f6e0ed56e4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.989007 28798 ts_tablet_manager.cc:1403] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.001s
I20260618 13:59:24.989153 28799 raft_consensus.cc:385] T c84189a4b1194c87bbdf81f6e0ed56e4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.989291 28799 raft_consensus.cc:740] T c84189a4b1194c87bbdf81f6e0ed56e4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.989648 28799 consensus_queue.cc:260] T c84189a4b1194c87bbdf81f6e0ed56e4 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:24.989893 28798 raft_consensus.cc:359] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.990074 28798 raft_consensus.cc:385] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:24.990159 28798 raft_consensus.cc:740] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:24.990372 28798 consensus_queue.cc:260] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.990870 28798 ts_tablet_manager.cc:1434] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:24.991173 28798 tablet_bootstrap.cc:492] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:24.992300 28799 ts_tablet_manager.cc:1434] T c84189a4b1194c87bbdf81f6e0ed56e4 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:24.992703 28799 tablet_bootstrap.cc:492] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:24.994371 28377 tablet_service.cc:1558] Processing DeleteTablet for tablet c84189a4b1194c87bbdf81f6e0ed56e4 with delete_type TABLET_DATA_DELETED (replaced by tablet be00bfa9bc1a4b83b18bfad05ba1495e) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:24.996932 28799 tablet_bootstrap.cc:654] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:24.998178 28806 raft_consensus.cc:493] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:24.998347 28806 raft_consensus.cc:515] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:24.999023 28806 leader_election.cc:290] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:24.999758 28617 leader_election.cc:343] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: d980cda1c60e4edca7b4d7537f4f5837
I20260618 13:59:25.000509 28801 ts_tablet_manager.cc:1929] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260618 13:59:25.000572 28618 leader_election.cc:343] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Not found: Tablet not found: d980cda1c60e4edca7b4d7537f4f5837
I20260618 13:59:25.000721 28801 log.cc:1199] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/ffd7d5ebaa94416a8eaf3e61bed945e1
I20260618 13:59:25.000696 28618 leader_election.cc:304] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:25.001121 28801 ts_tablet_manager.cc:1950] T ffd7d5ebaa94416a8eaf3e61bed945e1 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:25.001159 28806 raft_consensus.cc:2749] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:25.002527 28802 ts_tablet_manager.cc:1929] T 94365c44b5e24acab5df60c67203794a P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.004832 28802 log.cc:1199] T 94365c44b5e24acab5df60c67203794a P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/94365c44b5e24acab5df60c67203794a
I20260618 13:59:25.005416 28802 ts_tablet_manager.cc:1950] T 94365c44b5e24acab5df60c67203794a P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:25.001994 28798 tablet_bootstrap.cc:654] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.006790 28377 tablet_service.cc:1511] Processing CreateTablet for tablet 8ed88c142d7d40cda6392c06bc0486ca (DEFAULT_TABLE table=default.Table_ddb8f9243a5d47d1be63b3a371ca17ac [id=225463c2467047edae05b73a3132638f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.007196 28377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ed88c142d7d40cda6392c06bc0486ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.009238 28801 tablet_replica.cc:333] T 1f0109536c24485c9934193f2ef8aeae P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:25.009505 28801 raft_consensus.cc:2243] T 1f0109536c24485c9934193f2ef8aeae P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.009727 28801 raft_consensus.cc:2272] T 1f0109536c24485c9934193f2ef8aeae P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:25.009913 28797 tablet_bootstrap.cc:654] T ffd7d5ebaa94416a8eaf3e61bed945e1 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.010708 28801 ts_tablet_manager.cc:1916] T 1f0109536c24485c9934193f2ef8aeae P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:25.011075 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 7572d17724d249739be1a1cb9468f909 (DEFAULT_TABLE table=default.Table_b583e9f1f2464ebe9df9b4663bb0a4f8 [id=67643f46e18742859d79b0608aeaad23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.011483 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7572d17724d249739be1a1cb9468f909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.017174 28384 tablet_service.cc:1511] Processing CreateTablet for tablet ab521028ac1a4d0da013112d76bb169f (DEFAULT_TABLE table=default.Table_3deacb2c9668491a85bbb09ae3f57d66 [id=431a660bfbed46248a603dc7781cf543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.017592 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab521028ac1a4d0da013112d76bb169f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.021507 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 94365c44b5e24acab5df60c67203794a (table default.Table_e4c2db87fb024027b4a6e457a8524f10 [id=770fade529a64ec29e70968dd6d6bb93]) successfully deleted
I20260618 13:59:25.022174 28802 tablet_replica.cc:333] T 8b59255b14df4158afe37cb8890e7415 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:25.022603 28802 raft_consensus.cc:2243] T 8b59255b14df4158afe37cb8890e7415 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.023059 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_a44222ec49e54499bbd3d2c256abc71c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:25.023286 28802 raft_consensus.cc:2272] T 8b59255b14df4158afe37cb8890e7415 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:25.024065 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a44222ec49e54499bbd3d2c256abc71c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:25.024418 28799 tablet_bootstrap.cc:492] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:25.024560 28799 ts_tablet_manager.cc:1403] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.005s
I20260618 13:59:25.024871 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_dfe94c1f858644c98dba7db12d019477"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:25.025219 28224 master_service.cc:960] ReplaceTablet: received request to replace tablet be59f988f0e34b9480f859cf88b9c223 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:25.025367 28799 raft_consensus.cc:359] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.025532 28799 raft_consensus.cc:385] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.025619 28799 raft_consensus.cc:740] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.025859 28227 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_cf80fd5bbdde4cfb855c8fbe860a15f3" } modify_external_catalogs: true
I20260618 13:59:25.025884 28799 consensus_queue.cc:260] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.025967 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_6c11bf62ed454c5091c40670eda57845"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:25.026578 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6c11bf62ed454c5091c40670eda57845 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:25.027139 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dfe94c1f858644c98dba7db12d019477 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:25.028836 28527 tablet_service.cc:1511] Processing CreateTablet for tablet 015494374645469785f1f978eaf0e9a5 (DEFAULT_TABLE table=default.Table_7699960fdee24242bf51bcfcbd6f68c2 [id=3b9ab803536344d3a59ff77b2b58ae31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.029371 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 015494374645469785f1f978eaf0e9a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.030681 28516 tablet_service.cc:1511] Processing CreateTablet for tablet 2df9e9e811ec4c2ea68467b1294bb62e (DEFAULT_TABLE table=default.Table_caa35bd6fefa4c5bab90b4b7f1ec48d9 [id=a508a0cb72474a748aacdfd918e3782a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.031028 28516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2df9e9e811ec4c2ea68467b1294bb62e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.031337 28799 ts_tablet_manager.cc:1434] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260618 13:59:25.031756 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 2df9e9e811ec4c2ea68467b1294bb62e (DEFAULT_TABLE table=default.Table_caa35bd6fefa4c5bab90b4b7f1ec48d9 [id=a508a0cb72474a748aacdfd918e3782a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.032558 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2df9e9e811ec4c2ea68467b1294bb62e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.034591 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet d980cda1c60e4edca7b4d7537f4f5837 with delete_type TABLET_DATA_DELETED (replaced by tablet 98b42d23832545ae81a7cdff19e4689f) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.035166 28800 tablet_replica.cc:333] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:25.035416 28800 raft_consensus.cc:2243] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.035614 28800 raft_consensus.cc:2272] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:25.035835 28799 tablet_bootstrap.cc:492] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:25.036676 28800 ts_tablet_manager.cc:1916] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:25.037914 28802 ts_tablet_manager.cc:1916] T 8b59255b14df4158afe37cb8890e7415 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:25.038085 28799 tablet_bootstrap.cc:654] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.042826 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet b1f734a3f29b42e591d5564e99292b5e with delete_type TABLET_DATA_DELETED (replaced by tablet e025b64380314546af8a1ab90c817825) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.047312 28800 ts_tablet_manager.cc:1929] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.047690 28800 log.cc:1199] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/d980cda1c60e4edca7b4d7537f4f5837
I20260618 13:59:25.048635 28800 ts_tablet_manager.cc:1950] T d980cda1c60e4edca7b4d7537f4f5837 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:25.049005 28801 ts_tablet_manager.cc:1929] T 1f0109536c24485c9934193f2ef8aeae P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.051892 28798 tablet_bootstrap.cc:492] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:25.053148 28801 log.cc:1199] T 1f0109536c24485c9934193f2ef8aeae P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/1f0109536c24485c9934193f2ef8aeae
I20260618 13:59:25.053445 28798 ts_tablet_manager.cc:1403] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.062s user 0.007s sys 0.008s
I20260618 13:59:25.054533 28801 ts_tablet_manager.cc:1950] T 1f0109536c24485c9934193f2ef8aeae P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:25.055974 28798 raft_consensus.cc:359] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.056445 28798 raft_consensus.cc:385] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.056617 28798 raft_consensus.cc:740] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.057677 28801 tablet_replica.cc:333] T e1a75029992645c5b942c0ec07862bb4 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:25.058992 28648 tablet_service.cc:1511] Processing CreateTablet for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (DEFAULT_TABLE table=default.Table_6ba7e86569384c888e4208d3414512ae [id=cf981768efab4f4c9cc5353ac9c053a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.059872 28648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09cbe2337dd14da1b8e0629ded7e7ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.057808 28798 consensus_queue.cc:260] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.071038 28798 ts_tablet_manager.cc:1434] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.017s user 0.002s sys 0.000s
I20260618 13:59:25.071560 28798 tablet_bootstrap.cc:492] T b0c9d4131ea24809ac081e7a73a21e1c P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:25.072088 28801 raft_consensus.cc:2243] T e1a75029992645c5b942c0ec07862bb4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.072295 28801 raft_consensus.cc:2272] T e1a75029992645c5b942c0ec07862bb4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:25.073084 28801 ts_tablet_manager.cc:1916] T e1a75029992645c5b942c0ec07862bb4 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:25.079617 28801 ts_tablet_manager.cc:1929] T e1a75029992645c5b942c0ec07862bb4 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.084522 28799 tablet_bootstrap.cc:492] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:25.084713 28799 ts_tablet_manager.cc:1403] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.049s user 0.006s sys 0.014s
I20260618 13:59:25.085453 28797 tablet_bootstrap.cc:492] T ffd7d5ebaa94416a8eaf3e61bed945e1 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:25.085497 28799 raft_consensus.cc:359] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.085711 28799 raft_consensus.cc:385] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.085795 28799 raft_consensus.cc:740] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.085947 28797 ts_tablet_manager.cc:1403] T ffd7d5ebaa94416a8eaf3e61bed945e1 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.202s user 0.003s sys 0.001s
I20260618 13:59:25.086045 28799 consensus_queue.cc:260] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.082410 28798 tablet_bootstrap.cc:654] T b0c9d4131ea24809ac081e7a73a21e1c P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.086611 28799 ts_tablet_manager.cc:1434] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:25.087478 28799 tablet_bootstrap.cc:492] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:25.091226 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet b739fa1847184290833e13a2515f3a3a with delete_type TABLET_DATA_DELETED (replaced by tablet 36b8a46a41e543c58a946b6764618c7d) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.092622 28801 log.cc:1199] T e1a75029992645c5b942c0ec07862bb4 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/e1a75029992645c5b942c0ec07862bb4
I20260618 13:59:25.092774 28799 tablet_bootstrap.cc:654] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.093639 28801 ts_tablet_manager.cc:1950] T e1a75029992645c5b942c0ec07862bb4 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:25.103477 28798 tablet_bootstrap.cc:492] T b0c9d4131ea24809ac081e7a73a21e1c P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:25.103740 28798 ts_tablet_manager.cc:1403] T b0c9d4131ea24809ac081e7a73a21e1c P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.032s user 0.005s sys 0.005s
I20260618 13:59:25.104730 28798 raft_consensus.cc:359] T b0c9d4131ea24809ac081e7a73a21e1c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.105044 28798 raft_consensus.cc:385] T b0c9d4131ea24809ac081e7a73a21e1c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.105194 28798 raft_consensus.cc:740] T b0c9d4131ea24809ac081e7a73a21e1c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.105512 28798 consensus_queue.cc:260] T b0c9d4131ea24809ac081e7a73a21e1c P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.106178 28798 ts_tablet_manager.cc:1434] T b0c9d4131ea24809ac081e7a73a21e1c P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:25.106629 28798 tablet_bootstrap.cc:492] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:25.108456 28801 tablet_replica.cc:333] T b1f734a3f29b42e591d5564e99292b5e P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:25.108716 28801 raft_consensus.cc:2243] T b1f734a3f29b42e591d5564e99292b5e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.108981 28801 raft_consensus.cc:2272] T b1f734a3f29b42e591d5564e99292b5e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:25.109763 28801 ts_tablet_manager.cc:1916] T b1f734a3f29b42e591d5564e99292b5e P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:25.112818 28799 tablet_bootstrap.cc:492] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:25.113209 28798 tablet_bootstrap.cc:654] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.113157 28799 ts_tablet_manager.cc:1403] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.026s user 0.009s sys 0.005s
I20260618 13:59:25.115147 28799 raft_consensus.cc:359] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.115432 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet 668c6c6f2034484ca5ccdd51c22dea1d with delete_type TABLET_DATA_DELETED (replaced by tablet 9da90366cb7442bb9f4bf3bab07db070) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.115504 28799 raft_consensus.cc:385] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.115710 28799 raft_consensus.cc:740] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.116515 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet d502d67826ff4d7a8b0f515e5d8b6d77 with delete_type TABLET_DATA_DELETED (replaced by tablet 657e55fa53c447babfe91b1582806e6b) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.117002 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet d502d67826ff4d7a8b0f515e5d8b6d77 with delete_type TABLET_DATA_DELETED (replaced by tablet 657e55fa53c447babfe91b1582806e6b) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.117460 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet d1baf86497d442dbbbddbc3498bebbee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.118103 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet d1baf86497d442dbbbddbc3498bebbee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.118391 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet bce7bf74b6d24d0bb43808b8a62cf1b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.118880 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet bce7bf74b6d24d0bb43808b8a62cf1b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.119282 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet bd6b0cdffd5a44b5b45b885f6e88eb9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.120028 28527 tablet_service.cc:1511] Processing CreateTablet for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (DEFAULT_TABLE table=default.Table_6ba7e86569384c888e4208d3414512ae [id=cf981768efab4f4c9cc5353ac9c053a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.119949 28799 consensus_queue.cc:260] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.120744 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09cbe2337dd14da1b8e0629ded7e7ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.121805 28799 ts_tablet_manager.cc:1434] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260618 13:59:25.122470 28799 tablet_bootstrap.cc:492] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:25.122634 28810 raft_consensus.cc:493] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:25.122781 28810 raft_consensus.cc:515] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.123376 28801 ts_tablet_manager.cc:1929] T b1f734a3f29b42e591d5564e99292b5e P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.123494 28801 log.cc:1199] T b1f734a3f29b42e591d5564e99292b5e P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/b1f734a3f29b42e591d5564e99292b5e
I20260618 13:59:25.124150 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "668c6c6f2034484ca5ccdd51c22dea1d" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:25.125150 28801 ts_tablet_manager.cc:1950] T b1f734a3f29b42e591d5564e99292b5e P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
W20260618 13:59:25.125451 28484 leader_election.cc:343] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:25.126173 28801 tablet_replica.cc:333] T b739fa1847184290833e13a2515f3a3a P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:25.123489 28810 leader_election.cc:290] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:25.126525 28801 raft_consensus.cc:2243] T b739fa1847184290833e13a2515f3a3a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.126801 28801 raft_consensus.cc:2272] T b739fa1847184290833e13a2515f3a3a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:25.127277 28483 leader_election.cc:343] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 668c6c6f2034484ca5ccdd51c22dea1d
I20260618 13:59:25.127411 28483 leader_election.cc:304] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:25.128157 28810 raft_consensus.cc:2749] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:25.129182 28798 tablet_bootstrap.cc:492] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:25.129504 28798 ts_tablet_manager.cc:1403] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.023s user 0.005s sys 0.001s
I20260618 13:59:25.131209 28799 tablet_bootstrap.cc:654] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.131001 28798 raft_consensus.cc:359] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.131438 28798 raft_consensus.cc:385] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.131629 28798 raft_consensus.cc:740] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.132120 28798 consensus_queue.cc:260] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.132493 28801 ts_tablet_manager.cc:1916] T b739fa1847184290833e13a2515f3a3a P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:25.135399 28798 ts_tablet_manager.cc:1434] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260618 13:59:25.138171 28798 tablet_bootstrap.cc:492] T 95db5ddaf5d5486988750efed5972eb6 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:25.141584 28798 tablet_bootstrap.cc:654] T 95db5ddaf5d5486988750efed5972eb6 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.143769 28801 ts_tablet_manager.cc:1929] T b739fa1847184290833e13a2515f3a3a P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.144889 28801 log.cc:1199] T b739fa1847184290833e13a2515f3a3a P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/b739fa1847184290833e13a2515f3a3a
I20260618 13:59:25.145324 28801 ts_tablet_manager.cc:1950] T b739fa1847184290833e13a2515f3a3a P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:25.146603 28799 tablet_bootstrap.cc:492] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:25.146961 28799 ts_tablet_manager.cc:1403] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.005s
I20260618 13:59:25.147120 28797 raft_consensus.cc:359] T ffd7d5ebaa94416a8eaf3e61bed945e1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.147312 28801 tablet_replica.cc:333] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:25.147395 28797 raft_consensus.cc:385] T ffd7d5ebaa94416a8eaf3e61bed945e1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.147557 28797 raft_consensus.cc:740] T ffd7d5ebaa94416a8eaf3e61bed945e1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.147962 28801 raft_consensus.cc:2243] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.147878 28797 consensus_queue.cc:260] T ffd7d5ebaa94416a8eaf3e61bed945e1 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.148448 28797 ts_tablet_manager.cc:1434] T ffd7d5ebaa94416a8eaf3e61bed945e1 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.062s user 0.000s sys 0.002s
I20260618 13:59:25.148588 28799 raft_consensus.cc:359] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.148980 28799 raft_consensus.cc:385] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.149139 28799 raft_consensus.cc:740] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.149622 28799 consensus_queue.cc:260] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.150894 28799 ts_tablet_manager.cc:1434] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 13:59:25.150993 28801 raft_consensus.cc:2272] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:25.151530 28799 tablet_bootstrap.cc:492] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:25.151732 28801 ts_tablet_manager.cc:1916] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:25.157399 28799 tablet_bootstrap.cc:654] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.158465 28801 ts_tablet_manager.cc:1929] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.158614 28801 log.cc:1199] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/668c6c6f2034484ca5ccdd51c22dea1d
I20260618 13:59:25.159004 28801 ts_tablet_manager.cc:1950] T 668c6c6f2034484ca5ccdd51c22dea1d P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:25.159900 28227 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_cf80fd5bbdde4cfb855c8fbe860a15f3" } modify_external_catalogs: true
I20260618 13:59:25.162950 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet c552d1316e3340f4bcfa39618ae309b8 with delete_type TABLET_DATA_DELETED (replaced by tablet fe750fc5d998448caeab0a6c808dfba7) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:25.166900 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_bbf208e6e39a47349ab50198f178974e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:25.167675 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bbf208e6e39a47349ab50198f178974e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:25.169386 28220 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet e6de1293572f4e75af3417f281b9a142
I20260618 13:59:25.169593 28801 tablet_replica.cc:333] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:25.169862 28801 raft_consensus.cc:2243] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.170043 28801 raft_consensus.cc:2272] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:25.171000 28801 ts_tablet_manager.cc:1916] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:25.175377 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet e6de1293572f4e75af3417f281b9a142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:25.175721 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet 2188ebf1727240cd886ba7662f8fcf43 with delete_type TABLET_DATA_DELETED (replaced by tablet c0a42aa779ae452e9420d730bf8d7d55) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:25.175721 28377 tablet_service.cc:1558] Processing DeleteTablet for tablet 4c2b924a096b479cad31c29bf7c99172 with delete_type TABLET_DATA_DELETED (replaced by tablet ab521028ac1a4d0da013112d76bb169f) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:25.176222 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 516421cd843b44769efa2b555697c065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:25.189384 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_5d485acde41448d984aa79fa58b1866b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:25.190085 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d485acde41448d984aa79fa58b1866b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:25.191071 28934 raft_consensus.cc:493] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:25.191248 28934 raft_consensus.cc:515] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.191772 28801 ts_tablet_manager.cc:1929] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.191916 28801 log.cc:1199] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/bd6b0cdffd5a44b5b45b885f6e88eb9d
I20260618 13:59:25.191915 28934 leader_election.cc:290] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:25.191918 28653 tablet_service.cc:1511] Processing CreateTablet for tablet 8ed88c142d7d40cda6392c06bc0486ca (DEFAULT_TABLE table=default.Table_ddb8f9243a5d47d1be63b3a371ca17ac [id=225463c2467047edae05b73a3132638f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.192405 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ed88c142d7d40cda6392c06bc0486ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.192435 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c552d1316e3340f4bcfa39618ae309b8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:25.192524 28801 ts_tablet_manager.cc:1950] T bd6b0cdffd5a44b5b45b885f6e88eb9d P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
W20260618 13:59:25.193027 28346 leader_election.cc:343] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:25.193461 28527 tablet_service.cc:1511] Processing CreateTablet for tablet 8ed88c142d7d40cda6392c06bc0486ca (DEFAULT_TABLE table=default.Table_ddb8f9243a5d47d1be63b3a371ca17ac [id=225463c2467047edae05b73a3132638f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.193887 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ed88c142d7d40cda6392c06bc0486ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.193956 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c552d1316e3340f4bcfa39618ae309b8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:25.194408 28346 leader_election.cc:343] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:25.194535 28346 leader_election.cc:304] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:25.194768 28934 raft_consensus.cc:2749] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:25.197664 28516 tablet_service.cc:1511] Processing CreateTablet for tablet 7572d17724d249739be1a1cb9468f909 (DEFAULT_TABLE table=default.Table_b583e9f1f2464ebe9df9b4663bb0a4f8 [id=67643f46e18742859d79b0608aeaad23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.197654 28662 tablet_service.cc:1511] Processing CreateTablet for tablet 7572d17724d249739be1a1cb9468f909 (DEFAULT_TABLE table=default.Table_b583e9f1f2464ebe9df9b4663bb0a4f8 [id=67643f46e18742859d79b0608aeaad23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.198005 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7572d17724d249739be1a1cb9468f909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.198017 28516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7572d17724d249739be1a1cb9468f909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.204053 28797 tablet_bootstrap.cc:492] T e1a75029992645c5b942c0ec07862bb4 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:25.205042 28516 tablet_service.cc:1511] Processing CreateTablet for tablet ab521028ac1a4d0da013112d76bb169f (DEFAULT_TABLE table=default.Table_3deacb2c9668491a85bbb09ae3f57d66 [id=431a660bfbed46248a603dc7781cf543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.205447 28516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab521028ac1a4d0da013112d76bb169f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.206297 28797 tablet_bootstrap.cc:654] T e1a75029992645c5b942c0ec07862bb4 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.210783 28653 tablet_service.cc:1511] Processing CreateTablet for tablet ab521028ac1a4d0da013112d76bb169f (DEFAULT_TABLE table=default.Table_3deacb2c9668491a85bbb09ae3f57d66 [id=431a660bfbed46248a603dc7781cf543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.211241 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab521028ac1a4d0da013112d76bb169f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.211417 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet ebf502f25e9b4342bfaa58f42db7f2f5 (table default.Table_c1cda1e770c24c7eb9ab6c1ad8a7e786 [id=19c30a68cf504a179bbb2a966b68445c]) successfully deleted
I20260618 13:59:25.211658 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 37fedd27109944e2b3c870418c117758 (table default.Table_e9f01ff23601409189a5923e3755f4f8 [id=97b635f1072a4d82a82753d618600d4a]) successfully deleted
W20260618 13:59:25.211917 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet eead369de58f40378254697cbbbc6958 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eead369de58f40378254697cbbbc6958 already in progress: creating tablet
W20260618 13:59:25.212147 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 516421cd843b44769efa2b555697c065 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 516421cd843b44769efa2b555697c065 already in progress: creating tablet
I20260618 13:59:25.212347 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet d980cda1c60e4edca7b4d7537f4f5837 (table default.Table_3bd5981b65e545459eb539217e8e3599 [id=7c39b3c432e944bca05f7b2070a79ff9]) successfully deleted
I20260618 13:59:25.212532 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet ffd7d5ebaa94416a8eaf3e61bed945e1 (table default.Table_c18d75e5991b440caf136e9248a572da [id=5a3cb5befb614a70b4ecf8ccfedf8ecd]) successfully deleted
W20260618 13:59:25.212746 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 6eea4d59935d40aa97dc10ec7469aa56 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6eea4d59935d40aa97dc10ec7469aa56 already in progress: creating tablet
W20260618 13:59:25.212941 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 8fd9fa9675234db2abff694a517c0ea8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8fd9fa9675234db2abff694a517c0ea8 already in progress: creating tablet
W20260618 13:59:25.213171 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet aef7c1f111634c98af046e73fa9ffdeb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aef7c1f111634c98af046e73fa9ffdeb already in progress: creating tablet
I20260618 13:59:25.227470 28798 tablet_bootstrap.cc:492] T 95db5ddaf5d5486988750efed5972eb6 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:25.227644 28798 ts_tablet_manager.cc:1403] T 95db5ddaf5d5486988750efed5972eb6 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.090s user 0.006s sys 0.016s
I20260618 13:59:25.228436 28798 raft_consensus.cc:359] T 95db5ddaf5d5486988750efed5972eb6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.228641 28798 raft_consensus.cc:385] T 95db5ddaf5d5486988750efed5972eb6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.228747 28798 raft_consensus.cc:740] T 95db5ddaf5d5486988750efed5972eb6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.228953 28798 consensus_queue.cc:260] T 95db5ddaf5d5486988750efed5972eb6 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.229452 28798 ts_tablet_manager.cc:1434] T 95db5ddaf5d5486988750efed5972eb6 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:25.229745 28798 tablet_bootstrap.cc:492] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:25.231516 28798 tablet_bootstrap.cc:654] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.237648 28797 tablet_bootstrap.cc:492] T e1a75029992645c5b942c0ec07862bb4 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:25.237824 28797 ts_tablet_manager.cc:1403] T e1a75029992645c5b942c0ec07862bb4 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.003s
I20260618 13:59:25.238658 28797 raft_consensus.cc:359] T e1a75029992645c5b942c0ec07862bb4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.238859 28797 raft_consensus.cc:385] T e1a75029992645c5b942c0ec07862bb4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.238946 28797 raft_consensus.cc:740] T e1a75029992645c5b942c0ec07862bb4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.239226 28797 consensus_queue.cc:260] T e1a75029992645c5b942c0ec07862bb4 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.239728 28797 ts_tablet_manager.cc:1434] T e1a75029992645c5b942c0ec07862bb4 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
W20260618 13:59:25.218178 28225 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 827cc0987bd541759e334fb620f04ed5 consensus state has no leader
I20260618 13:59:25.218561 28229 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 316f470ea82f42fca48e4e96d49c9574
W20260618 13:59:25.219705 28226 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 316110bd2d024c7599fa2933d87e559e has no consensus state
I20260618 13:59:25.219857 28227 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet aa8cf927bed9437086efd8e0f21ba26a
I20260618 13:59:25.222884 28221 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_da651ce781664b8a8dfe3f1ab5774740" } new_table_name: "default.Table_efb2b8e451084b7ba0de07e0a6e538c2" modify_external_catalogs: true
I20260618 13:59:25.223529 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_c528008fd94b4fc8b7341f5b23dff76b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:25.253907 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c528008fd94b4fc8b7341f5b23dff76b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:25.254959 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet aa8cf927bed9437086efd8e0f21ba26a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:25.244060 28224 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet be59f988f0e34b9480f859cf88b9c223
I20260618 13:59:25.255823 28224 catalog_manager.cc:6784] ReplaceTablet: tablet be59f988f0e34b9480f859cf88b9c223 deleted and replaced by tablet b968a8d4014341ca9c5964263721ac79
I20260618 13:59:25.258822 28797 tablet_bootstrap.cc:492] T 1f0109536c24485c9934193f2ef8aeae P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:25.260551 28797 tablet_bootstrap.cc:654] T 1f0109536c24485c9934193f2ef8aeae P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:25.263332 28221 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9ead386562eb4fd787c7aee776c2194f has no consensus state
W20260618 13:59:25.275152 28215 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 61994.8us
W20260618 13:59:25.276561 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet fddea8fba4da40d7a5780e6750caf5e9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fddea8fba4da40d7a5780e6750caf5e9 already in progress: creating tablet
I20260618 13:59:25.277158 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 1f0109536c24485c9934193f2ef8aeae (table default.Table_90dfb0d3edbf419bb198321df1f067dc [id=cfb4ca8c4e884ccc8602be52d0b39100]) successfully deleted
W20260618 13:59:25.277451 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet c557afc3abfa4ef188bf248e89f49d87 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c557afc3abfa4ef188bf248e89f49d87 already in progress: creating tablet
W20260618 13:59:25.279754 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 04dac931e62a448d99633878eb70b92f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 04dac931e62a448d99633878eb70b92f already in progress: creating tablet
W20260618 13:59:25.280646 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 2f3d2f236fd44ab8af0524ea282cfa94 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f3d2f236fd44ab8af0524ea282cfa94 already in progress: creating tablet
W20260618 13:59:25.281196 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 2f5a334d18ba4c06bb358fa86d8af80d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f5a334d18ba4c06bb358fa86d8af80d already in progress: creating tablet
I20260618 13:59:25.281356 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet be59f988f0e34b9480f859cf88b9c223 with delete_type TABLET_DATA_DELETED (replaced by tablet b968a8d4014341ca9c5964263721ac79) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:25.281596 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet 316f470ea82f42fca48e4e96d49c9574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:25.282876 28802 ts_tablet_manager.cc:1929] T 8b59255b14df4158afe37cb8890e7415 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.283022 28802 log.cc:1199] T 8b59255b14df4158afe37cb8890e7415 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/8b59255b14df4158afe37cb8890e7415
I20260618 13:59:25.283406 28802 ts_tablet_manager.cc:1950] T 8b59255b14df4158afe37cb8890e7415 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:25.284402 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 8b59255b14df4158afe37cb8890e7415 (table default.Table_74c0d9b4f3724e278144c5fc30d8c2d7 [id=3bcc877eddec4d00b3dc1760cc635b36]) successfully deleted
I20260618 13:59:25.285324 28798 tablet_bootstrap.cc:492] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:25.285483 28798 ts_tablet_manager.cc:1403] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.056s user 0.002s sys 0.006s
I20260618 13:59:25.286211 28798 raft_consensus.cc:359] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.286442 28798 raft_consensus.cc:385] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.286520 28798 raft_consensus.cc:740] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.286752 28798 consensus_queue.cc:260] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.290261 28797 tablet_bootstrap.cc:492] T 1f0109536c24485c9934193f2ef8aeae P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:25.290436 28797 ts_tablet_manager.cc:1403] T 1f0109536c24485c9934193f2ef8aeae P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.002s
I20260618 13:59:25.291249 28797 raft_consensus.cc:359] T 1f0109536c24485c9934193f2ef8aeae P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.291411 28798 ts_tablet_manager.cc:1434] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260618 13:59:25.291428 28797 raft_consensus.cc:385] T 1f0109536c24485c9934193f2ef8aeae P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.293844 28798 tablet_bootstrap.cc:492] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:25.295459 28798 tablet_bootstrap.cc:654] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.296692 28802 tablet_replica.cc:333] T d9c00003300246ada2a356e3e1cf684b P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:25.296885 28802 raft_consensus.cc:2243] T d9c00003300246ada2a356e3e1cf684b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.297052 28802 raft_consensus.cc:2272] T d9c00003300246ada2a356e3e1cf684b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:25.311017 28802 ts_tablet_manager.cc:1916] T d9c00003300246ada2a356e3e1cf684b P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:25.284701 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet b51d0851efb34eaa9e9e4be6b292ef62 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b51d0851efb34eaa9e9e4be6b292ef62 already in progress: creating tablet
W20260618 13:59:25.314813 28215 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet ab521028ac1a4d0da013112d76bb169f because the tablet was not found. No further retry: Not found: Tablet not found: ab521028ac1a4d0da013112d76bb169f
W20260618 13:59:25.315084 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 5c75278db2234b11bf95f1cf5cc76edd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5c75278db2234b11bf95f1cf5cc76edd already in progress: creating tablet
I20260618 13:59:25.315281 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet e1a75029992645c5b942c0ec07862bb4 (table default.Table_af2d1c7150e84f96807cda9b1dabd299 [id=bd90cbbd5592449480ef377189a9c2de]) successfully deleted
I20260618 13:59:25.315482 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet b1f734a3f29b42e591d5564e99292b5e (table default.Table_26e7d8c4f91144268a2a43e83992667d [id=803d5220ce1a4dcfaf27d93c574be2ac]) successfully deleted
I20260618 13:59:25.315785 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet b739fa1847184290833e13a2515f3a3a (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]) successfully deleted
I20260618 13:59:25.315986 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 668c6c6f2034484ca5ccdd51c22dea1d (table default.Table_a6146ff7daf149b4bca36d8f09293605 [id=aa7194d6b3b2419294e5aa57d0043a11]) successfully deleted
W20260618 13:59:25.316190 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d502d67826ff4d7a8b0f515e5d8b6d77 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d502d67826ff4d7a8b0f515e5d8b6d77 already in progress: creating tablet
W20260618 13:59:25.316394 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d1baf86497d442dbbbddbc3498bebbee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d1baf86497d442dbbbddbc3498bebbee already in progress: creating tablet
W20260618 13:59:25.316583 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet bce7bf74b6d24d0bb43808b8a62cf1b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bce7bf74b6d24d0bb43808b8a62cf1b7 already in progress: creating tablet
I20260618 13:59:25.316756 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet bd6b0cdffd5a44b5b45b885f6e88eb9d (table default.Table_f32d5142bcf64f5c9e94b015dd620e10 [id=cfb012a9137e481eb0e03145a6404788]) successfully deleted
I20260618 13:59:25.317482 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet d980cda1c60e4edca7b4d7537f4f5837 (table default.Table_3bd5981b65e545459eb539217e8e3599 [id=7c39b3c432e944bca05f7b2070a79ff9]) successfully deleted
W20260618 13:59:25.317688 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d502d67826ff4d7a8b0f515e5d8b6d77 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d502d67826ff4d7a8b0f515e5d8b6d77 already in progress: creating tablet
W20260618 13:59:25.317878 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d1baf86497d442dbbbddbc3498bebbee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d1baf86497d442dbbbddbc3498bebbee already in progress: creating tablet
W20260618 13:59:25.318092 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet bce7bf74b6d24d0bb43808b8a62cf1b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bce7bf74b6d24d0bb43808b8a62cf1b7 already in progress: creating tablet
I20260618 13:59:25.319621 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet e6de1293572f4e75af3417f281b9a142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.319744 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 with delete_type TABLET_DATA_DELETED (replaced by tablet ac58e939730c4af681c2f08bcfbb6602) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.320254 28801 tablet_replica.cc:333] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:25.320328 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet e6de1293572f4e75af3417f281b9a142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.320478 28801 raft_consensus.cc:2243] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.320695 28801 raft_consensus.cc:2272] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:25.320930 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet b0c9d4131ea24809ac081e7a73a21e1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.321532 28801 ts_tablet_manager.cc:1916] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:25.324201 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet 95db5ddaf5d5486988750efed5972eb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.325240 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet 316f470ea82f42fca48e4e96d49c9574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.326488 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet aa8cf927bed9437086efd8e0f21ba26a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.327488 28648 tablet_service.cc:1558] Processing DeleteTablet for tablet 316f470ea82f42fca48e4e96d49c9574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.327749 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet aa8cf927bed9437086efd8e0f21ba26a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.327823 28516 tablet_service.cc:1558] Processing DeleteTablet for tablet 3752f49c9fc64071a45e0ce4ff4cfef2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.327960 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet be59f988f0e34b9480f859cf88b9c223 with delete_type TABLET_DATA_DELETED (replaced by tablet b968a8d4014341ca9c5964263721ac79) from {username='slave'} at 127.0.0.1:43976
W20260618 13:59:25.328574 28215 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet 9b75ece59d924e3db49e002bf0faa89c (table default.Table_031e1a03030b45c99c1270d612008f91 [id=387d3ff1762749efad926181fe4c746a], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 9b75ece59d924e3db49e002bf0faa89c consensus state has no leader
I20260618 13:59:25.328703 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet ffd7d5ebaa94416a8eaf3e61bed945e1 with delete_type TABLET_DATA_DELETED (replaced by tablet d6871232860b41d489e34d95eb897241) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.329026 28800 tablet_replica.cc:333] T ffd7d5ebaa94416a8eaf3e61bed945e1 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:25.329931 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet e1a75029992645c5b942c0ec07862bb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.329885 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_334d2fffc20b4a5688939d1067192640"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:25.329227 28800 raft_consensus.cc:2243] T ffd7d5ebaa94416a8eaf3e61bed945e1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.330330 28800 raft_consensus.cc:2272] T ffd7d5ebaa94416a8eaf3e61bed945e1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:25.330459 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_334d2fffc20b4a5688939d1067192640 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:25.331207 28800 ts_tablet_manager.cc:1916] T ffd7d5ebaa94416a8eaf3e61bed945e1 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:25.332104 28227 master_service.cc:960] ReplaceTablet: received request to replace tablet dd2385e6752d49429422fc854539e9c6 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:25.334656 28224 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_3a9b1ea0c47e41babda533950a3d5ba8" } new_table_name: "default.Table_c70bc1c98b3b49d7aa3080904667732d" modify_external_catalogs: true
I20260618 13:59:25.335886 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet be59f988f0e34b9480f859cf88b9c223 with delete_type TABLET_DATA_DELETED (replaced by tablet b968a8d4014341ca9c5964263721ac79) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.336347 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_abe180e115a04052ae89cc92d3c1c1e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:25.336809 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_abe180e115a04052ae89cc92d3c1c1e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:25.340373 28224 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 512a9e63ba324a30b8d8c37e585ebdf8 consensus state has no leader
I20260618 13:59:25.343645 28801 ts_tablet_manager.cc:1929] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.344142 28800 ts_tablet_manager.cc:1929] T ffd7d5ebaa94416a8eaf3e61bed945e1 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.344587 28223 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_be74477d3cd94a6891d879db47d3efd1" } new_table_name: "default.Table_a7356ad0d2d04b9788d4d980093d0df4" modify_external_catalogs: true
I20260618 13:59:25.344991 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_0ec7ef7f4c6e48d5b196e45f78a4fbf2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:25.344337 28801 log.cc:1199] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/b1f3dc9c3d874ca4ae42fd5e6d84fcb0
W20260618 13:59:25.345489 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ec7ef7f4c6e48d5b196e45f78a4fbf2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:25.346549 28800 log.cc:1199] T ffd7d5ebaa94416a8eaf3e61bed945e1 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/ffd7d5ebaa94416a8eaf3e61bed945e1
I20260618 13:59:25.346967 28800 ts_tablet_manager.cc:1950] T ffd7d5ebaa94416a8eaf3e61bed945e1 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:25.346446 28801 ts_tablet_manager.cc:1950] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
W20260618 13:59:25.349813 28223 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fe750fc5d998448caeab0a6c808dfba7 consensus state has no leader
I20260618 13:59:25.350207 28800 tablet_replica.cc:333] T e1a75029992645c5b942c0ec07862bb4 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:25.350409 28801 tablet_replica.cc:333] T b0c9d4131ea24809ac081e7a73a21e1c P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:25.350947 28800 raft_consensus.cc:2243] T e1a75029992645c5b942c0ec07862bb4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.351207 28801 raft_consensus.cc:2243] T b0c9d4131ea24809ac081e7a73a21e1c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.351539 28801 raft_consensus.cc:2272] T b0c9d4131ea24809ac081e7a73a21e1c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:25.351675 28800 raft_consensus.cc:2272] T e1a75029992645c5b942c0ec07862bb4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:25.352553 28800 ts_tablet_manager.cc:1916] T e1a75029992645c5b942c0ec07862bb4 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:25.353214 28801 ts_tablet_manager.cc:1916] T b0c9d4131ea24809ac081e7a73a21e1c P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:25.358105 28384 tablet_service.cc:1511] Processing CreateTablet for tablet ab100ebbe61a4321b9e76c730e5d46ed (DEFAULT_TABLE table=default.Table_9cf9f72ae27b44bc9aa43303a7103214 [id=f50e59ae00344758833392841b7f4985]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.358563 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab100ebbe61a4321b9e76c730e5d46ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.358995 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 3dd0eaa64271489d92aa380df877f257 (DEFAULT_TABLE table=default.Table_fd6ef45aeec84b48b8fb7a54db4b4f81 [id=ca38f91f52db4c69a0fb3bc6b8f5cadc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.359323 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dd0eaa64271489d92aa380df877f257. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.291517 28797 raft_consensus.cc:740] T 1f0109536c24485c9934193f2ef8aeae P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.359889 28797 consensus_queue.cc:260] T 1f0109536c24485c9934193f2ef8aeae P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.360375 28797 ts_tablet_manager.cc:1434] T 1f0109536c24485c9934193f2ef8aeae P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.070s user 0.000s sys 0.001s
I20260618 13:59:25.360551 28801 ts_tablet_manager.cc:1929] T b0c9d4131ea24809ac081e7a73a21e1c P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.360678 28801 log.cc:1199] T b0c9d4131ea24809ac081e7a73a21e1c P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/b0c9d4131ea24809ac081e7a73a21e1c
I20260618 13:59:25.361017 28801 ts_tablet_manager.cc:1950] T b0c9d4131ea24809ac081e7a73a21e1c P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:25.361877 28797 tablet_bootstrap.cc:492] T b1f734a3f29b42e591d5564e99292b5e P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:25.363626 28797 tablet_bootstrap.cc:654] T b1f734a3f29b42e591d5564e99292b5e P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.368415 28800 ts_tablet_manager.cc:1929] T e1a75029992645c5b942c0ec07862bb4 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.368569 28800 log.cc:1199] T e1a75029992645c5b942c0ec07862bb4 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/e1a75029992645c5b942c0ec07862bb4
I20260618 13:59:25.370553 28801 tablet_replica.cc:333] T 95db5ddaf5d5486988750efed5972eb6 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:25.370698 28800 ts_tablet_manager.cc:1950] T e1a75029992645c5b942c0ec07862bb4 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:25.371212 28799 tablet_bootstrap.cc:492] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:25.371398 28799 ts_tablet_manager.cc:1403] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.220s user 0.020s sys 0.056s
I20260618 13:59:25.371440 28801 raft_consensus.cc:2243] T 95db5ddaf5d5486988750efed5972eb6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.371694 28801 raft_consensus.cc:2272] T 95db5ddaf5d5486988750efed5972eb6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:25.372399 28802 ts_tablet_manager.cc:1929] T d9c00003300246ada2a356e3e1cf684b P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.372335 28799 raft_consensus.cc:359] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.372598 28799 raft_consensus.cc:385] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.372715 28799 raft_consensus.cc:740] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.373167 28801 ts_tablet_manager.cc:1916] T 95db5ddaf5d5486988750efed5972eb6 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:25.375998 28799 consensus_queue.cc:260] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.377348 28227 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet dd2385e6752d49429422fc854539e9c6
I20260618 13:59:25.377754 28227 catalog_manager.cc:6784] ReplaceTablet: tablet dd2385e6752d49429422fc854539e9c6 deleted and replaced by tablet c38a2156a55e4e5eb91d101e9a61eecd
I20260618 13:59:25.374095 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 4e4c6537144d425f81fd1fafcd034b77 (DEFAULT_TABLE table=default.Table_122b5d1531834ee2aa6bbc6593efb740 [id=873af388ffca420d958310ef2b50795c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.379788 28801 ts_tablet_manager.cc:1929] T 95db5ddaf5d5486988750efed5972eb6 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.379895 28801 log.cc:1199] T 95db5ddaf5d5486988750efed5972eb6 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/95db5ddaf5d5486988750efed5972eb6
I20260618 13:59:25.380259 28801 ts_tablet_manager.cc:1950] T 95db5ddaf5d5486988750efed5972eb6 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:25.378307 28377 tablet_service.cc:1558] Processing DeleteTablet for tablet dd2385e6752d49429422fc854539e9c6 with delete_type TABLET_DATA_DELETED (replaced by tablet c38a2156a55e4e5eb91d101e9a61eecd) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:25.381827 28797 tablet_bootstrap.cc:492] T b1f734a3f29b42e591d5564e99292b5e P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:25.381999 28797 ts_tablet_manager.cc:1403] T b1f734a3f29b42e591d5564e99292b5e P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.004s
I20260618 13:59:25.382205 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e4c6537144d425f81fd1fafcd034b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.383266 28797 raft_consensus.cc:359] T b1f734a3f29b42e591d5564e99292b5e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.383466 28797 raft_consensus.cc:385] T b1f734a3f29b42e591d5564e99292b5e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.383543 28797 raft_consensus.cc:740] T b1f734a3f29b42e591d5564e99292b5e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.383774 28797 consensus_queue.cc:260] T b1f734a3f29b42e591d5564e99292b5e P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.384277 28797 ts_tablet_manager.cc:1434] T b1f734a3f29b42e591d5564e99292b5e P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:25.384598 28797 tablet_bootstrap.cc:492] T bd6b0cdffd5a44b5b45b885f6e88eb9d P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:25.372514 28802 log.cc:1199] T d9c00003300246ada2a356e3e1cf684b P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/d9c00003300246ada2a356e3e1cf684b
I20260618 13:59:25.386163 28802 ts_tablet_manager.cc:1950] T d9c00003300246ada2a356e3e1cf684b P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:25.386189 28797 tablet_bootstrap.cc:654] T bd6b0cdffd5a44b5b45b885f6e88eb9d P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.386942 28227 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 1f81428f804a485c967026c4527fd250 (table default.Table_706eec6806a94d3f8d5b60708a0c1cda [id=b817db93d50144d7b7c238550eb53b10]). Expected version 1 got 0
I20260618 13:59:25.389151 28799 ts_tablet_manager.cc:1434] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.018s user 0.000s sys 0.002s
I20260618 13:59:25.391180 28227 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 1f81428f804a485c967026c4527fd250 (table default.Table_706eec6806a94d3f8d5b60708a0c1cda [id=b817db93d50144d7b7c238550eb53b10]). Expected version 1 got 0
I20260618 13:59:25.394624 28799 tablet_bootstrap.cc:492] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:25.396879 28377 tablet_service.cc:1511] Processing CreateTablet for tablet 1d8b3cfac0f648cd8b1c51102132c676 (DEFAULT_TABLE table=default.Table_644a49408bd64f4d98eaa6650b3b93e3 [id=b3649400476d486f8ec75e38db1480c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.397034 28799 tablet_bootstrap.cc:654] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.397071 28797 tablet_bootstrap.cc:492] T bd6b0cdffd5a44b5b45b885f6e88eb9d P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:25.397213 28797 ts_tablet_manager.cc:1403] T bd6b0cdffd5a44b5b45b885f6e88eb9d P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.007s
I20260618 13:59:25.397281 28377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d8b3cfac0f648cd8b1c51102132c676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.397926 28797 raft_consensus.cc:359] T bd6b0cdffd5a44b5b45b885f6e88eb9d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.398090 28797 raft_consensus.cc:385] T bd6b0cdffd5a44b5b45b885f6e88eb9d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.398190 28797 raft_consensus.cc:740] T bd6b0cdffd5a44b5b45b885f6e88eb9d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.398409 28797 consensus_queue.cc:260] T bd6b0cdffd5a44b5b45b885f6e88eb9d P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.398952 28797 ts_tablet_manager.cc:1434] T bd6b0cdffd5a44b5b45b885f6e88eb9d P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:25.399281 28797 tablet_bootstrap.cc:492] T b739fa1847184290833e13a2515f3a3a P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:25.401285 28797 tablet_bootstrap.cc:654] T b739fa1847184290833e13a2515f3a3a P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.403367 28801 tablet_replica.cc:333] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:25.403569 28801 raft_consensus.cc:2243] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.403728 28801 raft_consensus.cc:2272] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:25.403733 28391 tablet_service.cc:1511] Processing CreateTablet for tablet d464dd402d2044469e364ed37fdfb39b (DEFAULT_TABLE table=default.Table_e8ae99dfd8e84d7bb2723eb5fe129c2f [id=6d8f3597814241f3bc8c920d3a59008e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.404089 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d464dd402d2044469e364ed37fdfb39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.404472 28801 ts_tablet_manager.cc:1916] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:25.411247 28221 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_1b32156053d845a5b192d2ccf583cac6" } modify_external_catalogs: true
I20260618 13:59:25.411455 28221 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_1b32156053d845a5b192d2ccf583cac6" } modify_external_catalogs: true
I20260618 13:59:25.411552 28227 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_6ad6cf4da29143ff9ad3187cfcb95f6e" } modify_external_catalogs: true
I20260618 13:59:25.411741 28227 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_6ad6cf4da29143ff9ad3187cfcb95f6e" } modify_external_catalogs: true
I20260618 13:59:25.412633 28223 master_service.cc:960] ReplaceTablet: received request to replace tablet b6dd422c05324c989e29b20ae2afabad from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:25.413619 28224 master_service.cc:960] ReplaceTablet: received request to replace tablet 0463fa2e30554abcbfc8f37b702e81d4 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:25.414562 28226 master_service.cc:960] ReplaceTablet: received request to replace tablet fe750fc5d998448caeab0a6c808dfba7 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:25.420462 28518 tablet_service.cc:1511] Processing CreateTablet for tablet 4e4c6537144d425f81fd1fafcd034b77 (DEFAULT_TABLE table=default.Table_122b5d1531834ee2aa6bbc6593efb740 [id=873af388ffca420d958310ef2b50795c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.420565 28515 tablet_service.cc:1511] Processing CreateTablet for tablet 1d8b3cfac0f648cd8b1c51102132c676 (DEFAULT_TABLE table=default.Table_644a49408bd64f4d98eaa6650b3b93e3 [id=b3649400476d486f8ec75e38db1480c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.420882 28518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e4c6537144d425f81fd1fafcd034b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.420840 28653 tablet_service.cc:1511] Processing CreateTablet for tablet ab100ebbe61a4321b9e76c730e5d46ed (DEFAULT_TABLE table=default.Table_9cf9f72ae27b44bc9aa43303a7103214 [id=f50e59ae00344758833392841b7f4985]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.421221 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab100ebbe61a4321b9e76c730e5d46ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.421165 28521 tablet_service.cc:1511] Processing CreateTablet for tablet d464dd402d2044469e364ed37fdfb39b (DEFAULT_TABLE table=default.Table_e8ae99dfd8e84d7bb2723eb5fe129c2f [id=6d8f3597814241f3bc8c920d3a59008e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.421478 28521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d464dd402d2044469e364ed37fdfb39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.422577 28515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d8b3cfac0f648cd8b1c51102132c676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.424187 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet d9c00003300246ada2a356e3e1cf684b (table default.Table_a9703f9d3b54453691c354131f824e22 [id=f298b441bd9b444b9edd832be0adb03a]) successfully deleted
W20260618 13:59:25.424876 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet d502d67826ff4d7a8b0f515e5d8b6d77 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d502d67826ff4d7a8b0f515e5d8b6d77 already in progress: creating tablet
W20260618 13:59:25.425875 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet d1baf86497d442dbbbddbc3498bebbee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d1baf86497d442dbbbddbc3498bebbee already in progress: creating tablet
I20260618 13:59:25.426138 28802 tablet_replica.cc:333] T 36b8a46a41e543c58a946b6764618c7d P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:25.426348 28802 raft_consensus.cc:2243] T 36b8a46a41e543c58a946b6764618c7d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.426525 28802 raft_consensus.cc:2272] T 36b8a46a41e543c58a946b6764618c7d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:25.426723 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 3dd0eaa64271489d92aa380df877f257 (DEFAULT_TABLE table=default.Table_fd6ef45aeec84b48b8fb7a54db4b4f81 [id=ca38f91f52db4c69a0fb3bc6b8f5cadc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.427112 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dd0eaa64271489d92aa380df877f257. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.427299 28802 ts_tablet_manager.cc:1916] T 36b8a46a41e543c58a946b6764618c7d P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:25.428423 28221 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 18d8b609b96a4e81b57f69bff522eeea
I20260618 13:59:25.430380 28377 tablet_service.cc:1558] Processing DeleteTablet for tablet 18d8b609b96a4e81b57f69bff522eeea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:25.430469 28648 tablet_service.cc:1511] Processing CreateTablet for tablet 4e4c6537144d425f81fd1fafcd034b77 (DEFAULT_TABLE table=default.Table_122b5d1531834ee2aa6bbc6593efb740 [id=873af388ffca420d958310ef2b50795c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.430840 28648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e4c6537144d425f81fd1fafcd034b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.434948 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f0109536c24485c9934193f2ef8aeae with delete_type TABLET_DATA_DELETED (replaced by tablet 6b8f273def3e4b7caa4d624cf87a106c) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.435196 28800 tablet_replica.cc:333] T 1f0109536c24485c9934193f2ef8aeae P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:25.435377 28800 raft_consensus.cc:2243] T 1f0109536c24485c9934193f2ef8aeae P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.435531 28800 raft_consensus.cc:2272] T 1f0109536c24485c9934193f2ef8aeae P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:25.435539 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 (table default.Table_ceb33d60d86e4fd898ae67c78da94a55 [id=3251ae257e0f49b99352dcc1ef004588]) successfully deleted
W20260618 13:59:25.435755 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet e6de1293572f4e75af3417f281b9a142 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e6de1293572f4e75af3417f281b9a142 already in progress: creating tablet
I20260618 13:59:25.435974 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet b0c9d4131ea24809ac081e7a73a21e1c (table default.Table_ddcc7a2b28ca4095be58d4c859fd964a [id=0bd492e029e346f68f2db501d798ace1]) successfully deleted
I20260618 13:59:25.436152 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 95db5ddaf5d5486988750efed5972eb6 (table default.Table_067483ebd76c49e595543cc25ff1db56 [id=5dadba9b472745259796cf9015e88c65]) successfully deleted
I20260618 13:59:25.436259 28800 ts_tablet_manager.cc:1916] T 1f0109536c24485c9934193f2ef8aeae P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:25.436355 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 316f470ea82f42fca48e4e96d49c9574 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 316f470ea82f42fca48e4e96d49c9574 already in progress: creating tablet
W20260618 13:59:25.436558 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet aa8cf927bed9437086efd8e0f21ba26a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa8cf927bed9437086efd8e0f21ba26a already in progress: creating tablet
W20260618 13:59:25.436764 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet e6de1293572f4e75af3417f281b9a142 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e6de1293572f4e75af3417f281b9a142 already in progress: creating tablet
I20260618 13:59:25.436839 28651 tablet_service.cc:1511] Processing CreateTablet for tablet 1d8b3cfac0f648cd8b1c51102132c676 (DEFAULT_TABLE table=default.Table_644a49408bd64f4d98eaa6650b3b93e3 [id=b3649400476d486f8ec75e38db1480c3]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:25.436951 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 316f470ea82f42fca48e4e96d49c9574 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 316f470ea82f42fca48e4e96d49c9574 already in progress: creating tablet
I20260618 13:59:25.437170 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d8b3cfac0f648cd8b1c51102132c676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.437750 28226 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet fe750fc5d998448caeab0a6c808dfba7
I20260618 13:59:25.438238 28226 catalog_manager.cc:6784] ReplaceTablet: tablet fe750fc5d998448caeab0a6c808dfba7 deleted and replaced by tablet 438663f365e34fab87e34f434920cecf
I20260618 13:59:25.439491 28377 tablet_service.cc:1558] Processing DeleteTablet for tablet fe750fc5d998448caeab0a6c808dfba7 with delete_type TABLET_DATA_DELETED (replaced by tablet 438663f365e34fab87e34f434920cecf) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:25.440389 28377 tablet_service.cc:1511] Processing CreateTablet for tablet 18d8b609b96a4e81b57f69bff522eeea (DEFAULT_TABLE table=default.Table_1b32156053d845a5b192d2ccf583cac6 [id=60305da883dd4da2ab74ac31d5462bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.440459 28391 tablet_service.cc:1511] Processing CreateTablet for tablet f54910015987484e95b295352d859315 (DEFAULT_TABLE table=default.Table_d6f2768172e8455882ce01051fc835a4 [id=2c1b442aeecf44ee8b6140cacdb37627]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.440907 28227 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 7d4144b1b5ea4e968672616bc22bead8
I20260618 13:59:25.442168 28388 tablet_service.cc:1511] Processing CreateTablet for tablet 4bc87770baa34842a25ed97a19354446 (DEFAULT_TABLE table=default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.442322 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet 7d4144b1b5ea4e968672616bc22bead8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:25.442602 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet b1f734a3f29b42e591d5564e99292b5e with delete_type TABLET_DATA_DELETED (replaced by tablet e025b64380314546af8a1ab90c817825) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.442737 28800 ts_tablet_manager.cc:1929] T 1f0109536c24485c9934193f2ef8aeae P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.442853 28800 log.cc:1199] T 1f0109536c24485c9934193f2ef8aeae P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/1f0109536c24485c9934193f2ef8aeae
W20260618 13:59:25.442893 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet aa8cf927bed9437086efd8e0f21ba26a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa8cf927bed9437086efd8e0f21ba26a already in progress: creating tablet
I20260618 13:59:25.442986 28663 tablet_service.cc:1511] Processing CreateTablet for tablet d464dd402d2044469e364ed37fdfb39b (DEFAULT_TABLE table=default.Table_e8ae99dfd8e84d7bb2723eb5fe129c2f [id=6d8f3597814241f3bc8c920d3a59008e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:25.443099 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet be59f988f0e34b9480f859cf88b9c223 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet be59f988f0e34b9480f859cf88b9c223 already in progress: creating tablet
I20260618 13:59:25.443467 28934 raft_consensus.cc:493] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:25.443617 28934 raft_consensus.cc:515] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.444262 28934 leader_election.cc:290] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:25.442201 28385 tablet_service.cc:1511] Processing CreateTablet for tablet 9ead386562eb4fd787c7aee776c2194f (DEFAULT_TABLE table=default.Table_efb2b8e451084b7ba0de07e0a6e538c2 [id=333c0fd3837b4d89a9197eb6a826c311]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.445070 28384 tablet_service.cc:1511] Processing CreateTablet for tablet 56650c9e20e24a28821cca4279bfde58 (DEFAULT_TABLE table=default.Table_e24e443c392040b0adf1e8c8eb581a10 [id=b97c43b351b946c48f356b1ea1134239]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.443856 28223 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet b6dd422c05324c989e29b20ae2afabad
I20260618 13:59:25.445822 28223 catalog_manager.cc:6784] ReplaceTablet: tablet b6dd422c05324c989e29b20ae2afabad deleted and replaced by tablet 97a9f949999446f09359629fe483bf67
I20260618 13:59:25.443351 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d464dd402d2044469e364ed37fdfb39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.444046 28224 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 0463fa2e30554abcbfc8f37b702e81d4
I20260618 13:59:25.445492 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet ffd7d5ebaa94416a8eaf3e61bed945e1 (table default.Table_c18d75e5991b440caf136e9248a572da [id=5a3cb5befb614a70b4ecf8ccfedf8ecd]) successfully deleted
I20260618 13:59:25.446673 28224 catalog_manager.cc:6784] ReplaceTablet: tablet 0463fa2e30554abcbfc8f37b702e81d4 deleted and replaced by tablet fe1139c409e84070a2f496a85545e8d5
I20260618 13:59:25.446751 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet e1a75029992645c5b942c0ec07862bb4 (table default.Table_af2d1c7150e84f96807cda9b1dabd299 [id=bd90cbbd5592449480ef377189a9c2de]) successfully deleted
I20260618 13:59:25.447701 28661 tablet_service.cc:1558] Processing DeleteTablet for tablet fe750fc5d998448caeab0a6c808dfba7 with delete_type TABLET_DATA_DELETED (replaced by tablet 438663f365e34fab87e34f434920cecf) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.447937 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 18d8b609b96a4e81b57f69bff522eeea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.449009 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 7d4144b1b5ea4e968672616bc22bead8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.449373 28659 tablet_service.cc:1511] Processing CreateTablet for tablet 4bc87770baa34842a25ed97a19354446 (DEFAULT_TABLE table=default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.450145 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bc87770baa34842a25ed97a19354446. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.450798 28799 tablet_bootstrap.cc:492] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:25.451018 28799 ts_tablet_manager.cc:1403] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.056s user 0.004s sys 0.001s
I20260618 13:59:25.443450 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet bd6b0cdffd5a44b5b45b885f6e88eb9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.451937 28799 raft_consensus.cc:359] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.452173 28799 raft_consensus.cc:385] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.452250 28799 raft_consensus.cc:740] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.452534 28799 consensus_queue.cc:260] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.443218 28800 ts_tablet_manager.cc:1950] T 1f0109536c24485c9934193f2ef8aeae P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:25.453799 28650 tablet_service.cc:1511] Processing CreateTablet for tablet 56650c9e20e24a28821cca4279bfde58 (DEFAULT_TABLE table=default.Table_e24e443c392040b0adf1e8c8eb581a10 [id=b97c43b351b946c48f356b1ea1134239]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.454082 28800 tablet_replica.cc:333] T b1f734a3f29b42e591d5564e99292b5e P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:25.454162 28650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56650c9e20e24a28821cca4279bfde58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.454259 28800 raft_consensus.cc:2243] T b1f734a3f29b42e591d5564e99292b5e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.454449 28800 raft_consensus.cc:2272] T b1f734a3f29b42e591d5564e99292b5e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:25.456579 28799 ts_tablet_manager.cc:1434] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260618 13:59:25.458547 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet b6dd422c05324c989e29b20ae2afabad with delete_type TABLET_DATA_DELETED (replaced by tablet 97a9f949999446f09359629fe483bf67) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.458734 28658 tablet_service.cc:1558] Processing DeleteTablet for tablet 0463fa2e30554abcbfc8f37b702e81d4 with delete_type TABLET_DATA_DELETED (replaced by tablet fe1139c409e84070a2f496a85545e8d5) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.459334 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet dd2385e6752d49429422fc854539e9c6 with delete_type TABLET_DATA_DELETED (replaced by tablet c38a2156a55e4e5eb91d101e9a61eecd) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.443665 28649 tablet_service.cc:1511] Processing CreateTablet for tablet f54910015987484e95b295352d859315 (DEFAULT_TABLE table=default.Table_d6f2768172e8455882ce01051fc835a4 [id=2c1b442aeecf44ee8b6140cacdb37627]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.459875 28649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f54910015987484e95b295352d859315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.462873 28661 tablet_service.cc:1511] Processing CreateTablet for tablet 9ead386562eb4fd787c7aee776c2194f (DEFAULT_TABLE table=default.Table_efb2b8e451084b7ba0de07e0a6e538c2 [id=333c0fd3837b4d89a9197eb6a826c311]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.463249 28661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ead386562eb4fd787c7aee776c2194f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.466672 28377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18d8b609b96a4e81b57f69bff522eeea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.444567 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet dd2385e6752d49429422fc854539e9c6 with delete_type TABLET_DATA_DELETED (replaced by tablet c38a2156a55e4e5eb91d101e9a61eecd) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.445098 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da4d4f03911f4ac099ecc2fa1d8af494" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:25.467336 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f54910015987484e95b295352d859315. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:25.467573 28346 leader_election.cc:343] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:25.467790 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56650c9e20e24a28821cca4279bfde58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.467994 28655 tablet_service.cc:1511] Processing CreateTablet for tablet 18d8b609b96a4e81b57f69bff522eeea (DEFAULT_TABLE table=default.Table_1b32156053d845a5b192d2ccf583cac6 [id=60305da883dd4da2ab74ac31d5462bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.468354 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18d8b609b96a4e81b57f69bff522eeea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.470897 28934 raft_consensus.cc:493] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:25.471040 28934 raft_consensus.cc:515] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.471714 28934 leader_election.cc:290] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:25.472359 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188ebf1727240cd886ba7662f8fcf43" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:25.472955 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ead386562eb4fd787c7aee776c2194f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.473409 28799 tablet_bootstrap.cc:492] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:25.473474 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bc87770baa34842a25ed97a19354446. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.476284 28799 tablet_bootstrap.cc:654] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.478811 28934 raft_consensus.cc:493] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:25.479010 28934 raft_consensus.cc:515] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.479719 28934 leader_election.cc:290] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:25.479097 28346 leader_election.cc:343] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:25.480929 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1a06078974e4bfbbc4b949802a39bf1" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:25.481563 28346 leader_election.cc:343] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:25.497229 28802 ts_tablet_manager.cc:1929] T 36b8a46a41e543c58a946b6764618c7d P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.498075 28802 log.cc:1199] T 36b8a46a41e543c58a946b6764618c7d P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/36b8a46a41e543c58a946b6764618c7d
I20260618 13:59:25.501763 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 1f0109536c24485c9934193f2ef8aeae (table default.Table_90dfb0d3edbf419bb198321df1f067dc [id=cfb4ca8c4e884ccc8602be52d0b39100]) successfully deleted
I20260618 13:59:25.502002 28658 tablet_service.cc:1511] Processing CreateTablet for tablet 7aa9f65d8a5c46fdb745ee17af946dce (DEFAULT_TABLE table=default.Table_c2afd6643b7b49b7992384135658ddfc [id=29178416a03447608c28e809919d9461]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.502471 28658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7aa9f65d8a5c46fdb745ee17af946dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.504365 28802 ts_tablet_manager.cc:1950] T 36b8a46a41e543c58a946b6764618c7d P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:25.506333 28656 tablet_service.cc:1511] Processing CreateTablet for tablet 657e55fa53c447babfe91b1582806e6b (DEFAULT_TABLE table=default.Table_b5800aca80594ec1bd3a86ec47dbfc64 [id=2710734517a24ad6baf0d47bf23e9bdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.506822 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 657e55fa53c447babfe91b1582806e6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.507071 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 36b8a46a41e543c58a946b6764618c7d (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]) successfully deleted
I20260618 13:59:25.508152 28382 tablet_service.cc:1558] Processing DeleteTablet for tablet 0463fa2e30554abcbfc8f37b702e81d4 with delete_type TABLET_DATA_DELETED (replaced by tablet fe1139c409e84070a2f496a85545e8d5) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:25.508342 28380 tablet_service.cc:1511] Processing CreateTablet for tablet 657e55fa53c447babfe91b1582806e6b (DEFAULT_TABLE table=default.Table_b5800aca80594ec1bd3a86ec47dbfc64 [id=2710734517a24ad6baf0d47bf23e9bdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.508769 28380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 657e55fa53c447babfe91b1582806e6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.508710 28386 tablet_service.cc:1511] Processing CreateTablet for tablet 316110bd2d024c7599fa2933d87e559e (DEFAULT_TABLE table=default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.509148 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 316110bd2d024c7599fa2933d87e559e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.509377 28381 tablet_service.cc:1511] Processing CreateTablet for tablet 7aa9f65d8a5c46fdb745ee17af946dce (DEFAULT_TABLE table=default.Table_c2afd6643b7b49b7992384135658ddfc [id=29178416a03447608c28e809919d9461]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.509728 28381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7aa9f65d8a5c46fdb745ee17af946dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.512111 28802 tablet_replica.cc:333] T c84189a4b1194c87bbdf81f6e0ed56e4 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:25.512338 28802 raft_consensus.cc:2243] T c84189a4b1194c87bbdf81f6e0ed56e4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.512532 28802 raft_consensus.cc:2272] T c84189a4b1194c87bbdf81f6e0ed56e4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:25.506371 28662 tablet_service.cc:1511] Processing CreateTablet for tablet 316110bd2d024c7599fa2933d87e559e (DEFAULT_TABLE table=default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.513340 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 316110bd2d024c7599fa2933d87e559e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.513453 28802 ts_tablet_manager.cc:1916] T c84189a4b1194c87bbdf81f6e0ed56e4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:25.514655 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet b6dd422c05324c989e29b20ae2afabad with delete_type TABLET_DATA_DELETED (replaced by tablet 97a9f949999446f09359629fe483bf67) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:25.522889 28663 tablet_service.cc:1511] Processing CreateTablet for tablet ad045effb32948b5b8e9be5bb7996577 (DEFAULT_TABLE table=default.Table_7910db46cf394cdebe608e7efb3388cc [id=421bf4faf8174ccd88dc15799671c74d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.523257 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad045effb32948b5b8e9be5bb7996577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.523574 28223 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_0d8a6f9e3b84409e98be6ee8668907e2" } modify_external_catalogs: true
I20260618 13:59:25.523801 28223 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_0d8a6f9e3b84409e98be6ee8668907e2" } modify_external_catalogs: true
I20260618 13:59:25.525101 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_06cd6ce4fc4e4dcfb2349c87e228c7de"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:25.525650 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_06cd6ce4fc4e4dcfb2349c87e228c7de in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:25.527016 28652 tablet_service.cc:1511] Processing CreateTablet for tablet 538077e459a44ed78c95f2e881b6e0b0 (DEFAULT_TABLE table=default.Table_726d3825b282496fb87c67a8975e3fcc [id=f914bcce8551443caa218e7a92967307]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.527434 28652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 538077e459a44ed78c95f2e881b6e0b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.533704 28657 tablet_service.cc:1511] Processing CreateTablet for tablet 9ab1ddf79b094ba2aeee9225c0b332c3 (DEFAULT_TABLE table=default.Table_8e7925d0e7eb421685f09b8c53667351 [id=bbe67818f204445c89ede107910a3bdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.534063 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ab1ddf79b094ba2aeee9225c0b332c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.537599 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_9a78338c3e414b0abea1281082202f57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:25.538156 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9a78338c3e414b0abea1281082202f57 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:25.539495 28226 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_9cd59394f11d4937887809241a69bba3" } modify_external_catalogs: true
I20260618 13:59:25.539680 28226 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_9cd59394f11d4937887809241a69bba3" } modify_external_catalogs: true
I20260618 13:59:25.540954 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_aeeec1571f064ab1816cbf26635190fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:25.541059 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet bce7bf74b6d24d0bb43808b8a62cf1b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bce7bf74b6d24d0bb43808b8a62cf1b7 already in progress: creating tablet
W20260618 13:59:25.541476 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aeeec1571f064ab1816cbf26635190fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:25.542583 28229 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_2a3bcaddd3034d67a5ea7c78f0ea9e7c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:25.543076 28229 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2a3bcaddd3034d67a5ea7c78f0ea9e7c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:25.544353 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_3eaa556cd32f4cc1a275c041a0ca104b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:25.544844 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3eaa556cd32f4cc1a275c041a0ca104b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:25.546438 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_a7ff69763fdb4ff3b26b180bf05907d9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:25.546968 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a7ff69763fdb4ff3b26b180bf05907d9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:25.548115 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_2f8e4182bb594c2bad77994c12ce821f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:25.548549 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2f8e4182bb594c2bad77994c12ce821f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:25.550268 28225 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_6ba7e86569384c888e4208d3414512ae" } new_table_name: "default.Table_8f61f6724b1d44408a1d2efa16fe8a2d" modify_external_catalogs: true
I20260618 13:59:25.554004 28223 catalog_manager.cc:5978] Not sending DeleteTablet requests; no consensus state for tablet 29683aa706994d4181271c48b4efc0cd
I20260618 13:59:25.554354 28223 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_6c11bf62ed454c5091c40670eda57845" } new_table_name: "default.Table_13510793ddbc477cbdd8f32a8450fb45" modify_external_catalogs: true
I20260618 13:59:25.558898 28800 ts_tablet_manager.cc:1916] T b1f734a3f29b42e591d5564e99292b5e P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:25.564815 28226 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet b0285c75c632457383e81805ab036aaf
I20260618 13:59:25.565637 28226 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_f4a03a199e51400ca4c8a5d868ec2025" } modify_external_catalogs: true
I20260618 13:59:25.565840 28226 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_f4a03a199e51400ca4c8a5d868ec2025" } modify_external_catalogs: true
I20260618 13:59:25.567301 28797 tablet_bootstrap.cc:492] T b739fa1847184290833e13a2515f3a3a P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:25.567482 28797 ts_tablet_manager.cc:1403] T b739fa1847184290833e13a2515f3a3a P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.168s user 0.006s sys 0.000s
I20260618 13:59:25.567649 28229 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_142b5f36b1c5441595c29bf65d537a97" } new_table_name: "default.Table_98d3bc8b4a7044d685c351a61e02f3e3" modify_external_catalogs: true
I20260618 13:59:25.568490 28797 raft_consensus.cc:359] T b739fa1847184290833e13a2515f3a3a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.568707 28797 raft_consensus.cc:385] T b739fa1847184290833e13a2515f3a3a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.568778 28797 raft_consensus.cc:740] T b739fa1847184290833e13a2515f3a3a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.569078 28797 consensus_queue.cc:260] T b739fa1847184290833e13a2515f3a3a P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.569568 28797 ts_tablet_manager.cc:1434] T b739fa1847184290833e13a2515f3a3a P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:25.570003 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet b0285c75c632457383e81805ab036aaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.571914 28655 tablet_service.cc:1511] Processing CreateTablet for tablet 650ed7ee8b134bc4b9f95028c54f99be (DEFAULT_TABLE table=default.Table_b5feee88c56b42b6bd1b8ecb81a134bb [id=635bd8a93b22415fab4ee952247c6870]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.572398 28797 tablet_bootstrap.cc:492] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:25.574371 28797 tablet_bootstrap.cc:654] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.575008 28527 tablet_service.cc:1511] Processing CreateTablet for tablet ab100ebbe61a4321b9e76c730e5d46ed (DEFAULT_TABLE table=default.Table_9cf9f72ae27b44bc9aa43303a7103214 [id=f50e59ae00344758833392841b7f4985]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.575489 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab100ebbe61a4321b9e76c730e5d46ed. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:25.578934 28225 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 09cbe2337dd14da1b8e0629ded7e7ad0 consensus state has no leader
W20260618 13:59:25.581360 28229 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 957670718c8c4b8cb87115cc4a2fa861 consensus state has no leader
I20260618 13:59:25.582504 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da4d4f03911f4ac099ecc2fa1d8af494" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:25.583549 28226 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet d667ea5f029f46d89fcb3be03069b97b
I20260618 13:59:25.584780 28522 tablet_service.cc:1511] Processing CreateTablet for tablet f54910015987484e95b295352d859315 (DEFAULT_TABLE table=default.Table_d6f2768172e8455882ce01051fc835a4 [id=2c1b442aeecf44ee8b6140cacdb37627]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.583381 28516 tablet_service.cc:1511] Processing CreateTablet for tablet 3dd0eaa64271489d92aa380df877f257 (DEFAULT_TABLE table=default.Table_fd6ef45aeec84b48b8fb7a54db4b4f81 [id=ca38f91f52db4c69a0fb3bc6b8f5cadc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.586199 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f54910015987484e95b295352d859315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.587532 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet 18d8b609b96a4e81b57f69bff522eeea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.588510 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet fe750fc5d998448caeab0a6c808dfba7 with delete_type TABLET_DATA_DELETED (replaced by tablet 438663f365e34fab87e34f434920cecf) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.588829 28516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dd0eaa64271489d92aa380df877f257. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.588788 28511 tablet_service.cc:1511] Processing CreateTablet for tablet 18d8b609b96a4e81b57f69bff522eeea (DEFAULT_TABLE table=default.Table_1b32156053d845a5b192d2ccf583cac6 [id=60305da883dd4da2ab74ac31d5462bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.589134 28511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18d8b609b96a4e81b57f69bff522eeea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.589965 28517 tablet_service.cc:1511] Processing CreateTablet for tablet 9ead386562eb4fd787c7aee776c2194f (DEFAULT_TABLE table=default.Table_efb2b8e451084b7ba0de07e0a6e538c2 [id=333c0fd3837b4d89a9197eb6a826c311]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.590291 28517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ead386562eb4fd787c7aee776c2194f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:25.590377 28223 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet aa93f065b4d64e7b85e30f0690fae541 has no consensus state
I20260618 13:59:25.593344 28515 tablet_service.cc:1511] Processing CreateTablet for tablet 4bc87770baa34842a25ed97a19354446 (DEFAULT_TABLE table=default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.593680 28515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bc87770baa34842a25ed97a19354446. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.595705 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet 7d4144b1b5ea4e968672616bc22bead8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.595826 28518 tablet_service.cc:1511] Processing CreateTablet for tablet 56650c9e20e24a28821cca4279bfde58 (DEFAULT_TABLE table=default.Table_e24e443c392040b0adf1e8c8eb581a10 [id=b97c43b351b946c48f356b1ea1134239]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.596910 28518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56650c9e20e24a28821cca4279bfde58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.601558 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet b6dd422c05324c989e29b20ae2afabad with delete_type TABLET_DATA_DELETED (replaced by tablet 97a9f949999446f09359629fe483bf67) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.607216 28529 tablet_service.cc:1511] Processing CreateTablet for tablet 9ab1ddf79b094ba2aeee9225c0b332c3 (DEFAULT_TABLE table=default.Table_8e7925d0e7eb421685f09b8c53667351 [id=bbe67818f204445c89ede107910a3bdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.607743 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet 0463fa2e30554abcbfc8f37b702e81d4 with delete_type TABLET_DATA_DELETED (replaced by tablet fe1139c409e84070a2f496a85545e8d5) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.608245 28525 tablet_service.cc:1511] Processing CreateTablet for tablet 657e55fa53c447babfe91b1582806e6b (DEFAULT_TABLE table=default.Table_b5800aca80594ec1bd3a86ec47dbfc64 [id=2710734517a24ad6baf0d47bf23e9bdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.608253 28519 tablet_service.cc:1511] Processing CreateTablet for tablet 538077e459a44ed78c95f2e881b6e0b0 (DEFAULT_TABLE table=default.Table_726d3825b282496fb87c67a8975e3fcc [id=f914bcce8551443caa218e7a92967307]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.608435 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 316110bd2d024c7599fa2933d87e559e (DEFAULT_TABLE table=default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.610090 28513 tablet_service.cc:1511] Processing CreateTablet for tablet 15483938a06e459789f87c6bb456bd15 (DEFAULT_TABLE table=default.Table_f37d21a9a59249d6a0b87c3788ff4873 [id=22eb9037fc66472793e9fd3f03fd7218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.610944 28512 tablet_service.cc:1511] Processing CreateTablet for tablet 41c92a594cd64982a8972e4a019d1cd8 (DEFAULT_TABLE table=default.Table_387aa217e7f04d2f826dc1f82c256a97 [id=d5294ec922b94ef7b1a4394f00462ff4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.611078 28514 tablet_service.cc:1511] Processing CreateTablet for tablet 9024952ab8094fcda1329562bbde8ef5 (DEFAULT_TABLE table=default.Table_55f1c253e54246eda98a7dcf2996952b [id=5e62ed351f73400ab201c1b5dd43f28f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.611682 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188ebf1727240cd886ba7662f8fcf43" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:25.612831 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ab1ddf79b094ba2aeee9225c0b332c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.613562 28514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9024952ab8094fcda1329562bbde8ef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.614832 28510 tablet_service.cc:1511] Processing CreateTablet for tablet ad045effb32948b5b8e9be5bb7996577 (DEFAULT_TABLE table=default.Table_7910db46cf394cdebe608e7efb3388cc [id=421bf4faf8174ccd88dc15799671c74d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.615155 28520 tablet_service.cc:1511] Processing CreateTablet for tablet 650ed7ee8b134bc4b9f95028c54f99be (DEFAULT_TABLE table=default.Table_b5feee88c56b42b6bd1b8ecb81a134bb [id=635bd8a93b22415fab4ee952247c6870]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.618467 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1a06078974e4bfbbc4b949802a39bf1" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:25.618871 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 7aa9f65d8a5c46fdb745ee17af946dce (DEFAULT_TABLE table=default.Table_c2afd6643b7b49b7992384135658ddfc [id=29178416a03447608c28e809919d9461]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.624214 28226 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_f8635c3fdf8b4587bf0ac7791405eeac" } new_table_name: "default.Table_dbc6162c06f74e71b792bd1f7b2388fe" modify_external_catalogs: true
I20260618 13:59:25.625375 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet d667ea5f029f46d89fcb3be03069b97b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.625942 28524 tablet_service.cc:1511] Processing CreateTablet for tablet c5d046e0421142cd84c5b12d6a434f53 (DEFAULT_TABLE table=default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.629304 28528 tablet_service.cc:1511] Processing CreateTablet for tablet f3ba9186b3ff47d5a2b158fef619686e (DEFAULT_TABLE table=default.Table_3ef2ff183a6044e2a9e069a4eac133b8 [id=60e91b2138bf41a5a8dee65c1874678a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.631350 28513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15483938a06e459789f87c6bb456bd15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.632383 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet b0285c75c632457383e81805ab036aaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:25.635082 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_7fa3da42543d4978aac5ac383ed10056"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:25.635661 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7fa3da42543d4978aac5ac383ed10056 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:25.638587 28379 tablet_service.cc:1511] Processing CreateTablet for tablet 9ab1ddf79b094ba2aeee9225c0b332c3 (DEFAULT_TABLE table=default.Table_8e7925d0e7eb421685f09b8c53667351 [id=bbe67818f204445c89ede107910a3bdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.639011 28223 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_726d3825b282496fb87c67a8975e3fcc" } new_table_name: "default.Table_6911f12e0ef94ed8a9589f9294ccab4f" modify_external_catalogs: true
I20260618 13:59:25.640861 28229 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_9c526031302044508f50fead286b4465"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:25.641413 28229 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c526031302044508f50fead286b4465 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:25.642970 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_21633c55ca8c4614a2a0b241a0b89ee1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:25.643604 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_21633c55ca8c4614a2a0b241a0b89ee1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:25.652801 28226 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dc36fbea853a4d4e9f61909200239aed consensus state has no leader
I20260618 13:59:25.662194 28387 tablet_service.cc:1511] Processing CreateTablet for tablet 41c92a594cd64982a8972e4a019d1cd8 (DEFAULT_TABLE table=default.Table_387aa217e7f04d2f826dc1f82c256a97 [id=d5294ec922b94ef7b1a4394f00462ff4]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:25.663712 28346 leader_election.cc:343] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:25.664108 28390 tablet_service.cc:1511] Processing CreateTablet for tablet 9024952ab8094fcda1329562bbde8ef5 (DEFAULT_TABLE table=default.Table_55f1c253e54246eda98a7dcf2996952b [id=5e62ed351f73400ab201c1b5dd43f28f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.664707 28383 tablet_service.cc:1511] Processing CreateTablet for tablet c5d046e0421142cd84c5b12d6a434f53 (DEFAULT_TABLE table=default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.665822 28382 tablet_service.cc:1511] Processing CreateTablet for tablet ad045effb32948b5b8e9be5bb7996577 (DEFAULT_TABLE table=default.Table_7910db46cf394cdebe608e7efb3388cc [id=421bf4faf8174ccd88dc15799671c74d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.668668 28378 tablet_service.cc:1511] Processing CreateTablet for tablet 538077e459a44ed78c95f2e881b6e0b0 (DEFAULT_TABLE table=default.Table_726d3825b282496fb87c67a8975e3fcc [id=f914bcce8551443caa218e7a92967307]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.670708 28379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ab1ddf79b094ba2aeee9225c0b332c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.673154 28380 tablet_service.cc:1558] Processing DeleteTablet for tablet d667ea5f029f46d89fcb3be03069b97b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:25.674468 28799 tablet_bootstrap.cc:492] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:25.671769 28375 tablet_service.cc:1511] Processing CreateTablet for tablet 35d8dd2c388e43a0be380aa953ec2c82 (DEFAULT_TABLE table=default.Table_a8705c5cd49a40049ff1020a01aa7f5b [id=f9ff809bb5da4d048e7520cec3316f6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.674649 28799 ts_tablet_manager.cc:1403] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.201s user 0.004s sys 0.000s
I20260618 13:59:25.674189 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 650ed7ee8b134bc4b9f95028c54f99be (DEFAULT_TABLE table=default.Table_b5feee88c56b42b6bd1b8ecb81a134bb [id=635bd8a93b22415fab4ee952247c6870]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.674384 28376 tablet_service.cc:1511] Processing CreateTablet for tablet 957670718c8c4b8cb87115cc4a2fa861 (DEFAULT_TABLE table=default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.678258 28374 tablet_service.cc:1511] Processing CreateTablet for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (DEFAULT_TABLE table=default.Table_7cd820b0428a41ada19a2f14560a67e4 [id=2c73a9d87a914f03b09ef422d49f7a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.680099 28377 tablet_service.cc:1511] Processing CreateTablet for tablet f3ba9186b3ff47d5a2b158fef619686e (DEFAULT_TABLE table=default.Table_3ef2ff183a6044e2a9e069a4eac133b8 [id=60e91b2138bf41a5a8dee65c1874678a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.678146 28373 tablet_service.cc:1511] Processing CreateTablet for tablet 30f4bd3f86484caab48c1f0806299460 (DEFAULT_TABLE table=default.Table_6dc717aafef14b22ab95830a6cf50642 [id=0751d5ab69704387856ad7d40431e74b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.694191 28346 leader_election.cc:304] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:25.705479 28223 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 538077e459a44ed78c95f2e881b6e0b0 consensus state has no leader
I20260618 13:59:25.713860 28657 tablet_service.cc:1511] Processing CreateTablet for tablet 957670718c8c4b8cb87115cc4a2fa861 (DEFAULT_TABLE table=default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.719540 28663 tablet_service.cc:1511] Processing CreateTablet for tablet c5d046e0421142cd84c5b12d6a434f53 (DEFAULT_TABLE table=default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.721036 28800 ts_tablet_manager.cc:1929] T b1f734a3f29b42e591d5564e99292b5e P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.721705 28800 log.cc:1199] T b1f734a3f29b42e591d5564e99292b5e P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/b1f734a3f29b42e591d5564e99292b5e
I20260618 13:59:25.724170 28800 ts_tablet_manager.cc:1950] T b1f734a3f29b42e591d5564e99292b5e P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:25.600445 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet b739fa1847184290833e13a2515f3a3a with delete_type TABLET_DATA_DELETED (replaced by tablet 36b8a46a41e543c58a946b6764618c7d) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.710230 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet d667ea5f029f46d89fcb3be03069b97b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:25.712224 28662 tablet_service.cc:1511] Processing CreateTablet for tablet 35d8dd2c388e43a0be380aa953ec2c82 (DEFAULT_TABLE table=default.Table_a8705c5cd49a40049ff1020a01aa7f5b [id=f9ff809bb5da4d048e7520cec3316f6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.727123 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35d8dd2c388e43a0be380aa953ec2c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.727698 28223 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_99428babdde649b8b5d5fab801b58136" } modify_external_catalogs: true
I20260618 13:59:25.727917 28223 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_99428babdde649b8b5d5fab801b58136" } modify_external_catalogs: true
I20260618 13:59:25.712384 28654 tablet_service.cc:1511] Processing CreateTablet for tablet ca2ee746c5024cfc9b31c470f9616e7d (DEFAULT_TABLE table=default.Table_72e5809d8cce4f3bbe8227efcb8e2b2c [id=dcd71daa60ee430d9ca46a1463d9a1bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.730782 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca2ee746c5024cfc9b31c470f9616e7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.733577 28800 tablet_replica.cc:333] T bd6b0cdffd5a44b5b45b885f6e88eb9d P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:25.712610 28658 tablet_service.cc:1511] Processing CreateTablet for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (DEFAULT_TABLE table=default.Table_7cd820b0428a41ada19a2f14560a67e4 [id=2c73a9d87a914f03b09ef422d49f7a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.712672 28647 tablet_service.cc:1511] Processing CreateTablet for tablet fa24afe9db46408cace50f642f879f4b (DEFAULT_TABLE table=default.Table_4108c7ea79f5426395746346c44df521 [id=5a04afb74af64b17914f1587d7ddc542]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.712956 28652 tablet_service.cc:1511] Processing CreateTablet for tablet 30f4bd3f86484caab48c1f0806299460 (DEFAULT_TABLE table=default.Table_6dc717aafef14b22ab95830a6cf50642 [id=0751d5ab69704387856ad7d40431e74b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.734664 28800 raft_consensus.cc:2243] T bd6b0cdffd5a44b5b45b885f6e88eb9d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:25.713260 28656 tablet_service.cc:1511] Processing CreateTablet for tablet f9f7359c4a144ca89fe3e51d1365a425 (DEFAULT_TABLE table=default.Table_7d317fa28e3142a19d15c8360151c357 [id=bba3ab8821ca48a5aa1104094d78892a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.735641 28800 raft_consensus.cc:2272] T bd6b0cdffd5a44b5b45b885f6e88eb9d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:25.734755 28658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf5e04eb43ed4cf493c0f6f08e22baeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.737695 28647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa24afe9db46408cace50f642f879f4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.739008 28800 ts_tablet_manager.cc:1916] T bd6b0cdffd5a44b5b45b885f6e88eb9d P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:25.742761 28652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30f4bd3f86484caab48c1f0806299460. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.747354 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9f7359c4a144ca89fe3e51d1365a425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.683504 28372 tablet_service.cc:1511] Processing CreateTablet for tablet f9f7359c4a144ca89fe3e51d1365a425 (DEFAULT_TABLE table=default.Table_7d317fa28e3142a19d15c8360151c357 [id=bba3ab8821ca48a5aa1104094d78892a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.712683 28651 tablet_service.cc:1511] Processing CreateTablet for tablet 15483938a06e459789f87c6bb456bd15 (DEFAULT_TABLE table=default.Table_f37d21a9a59249d6a0b87c3788ff4873 [id=22eb9037fc66472793e9fd3f03fd7218]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:25.765269 28346 leader_election.cc:343] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:25.767758 28387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41c92a594cd64982a8972e4a019d1cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.768623 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 650ed7ee8b134bc4b9f95028c54f99be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.770556 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet b0285c75c632457383e81805ab036aaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:25.791152 28934 raft_consensus.cc:2749] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:25.816915 28799 raft_consensus.cc:359] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.817201 28799 raft_consensus.cc:385] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.820132 28512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41c92a594cd64982a8972e4a019d1cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.824638 28798 tablet_bootstrap.cc:492] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:25.827203 28380 tablet_service.cc:1511] Processing CreateTablet for tablet ca2ee746c5024cfc9b31c470f9616e7d (DEFAULT_TABLE table=default.Table_72e5809d8cce4f3bbe8227efcb8e2b2c [id=dcd71daa60ee430d9ca46a1463d9a1bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.824579 28521 tablet_service.cc:1511] Processing CreateTablet for tablet 957670718c8c4b8cb87115cc4a2fa861 (DEFAULT_TABLE table=default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.841300 28801 ts_tablet_manager.cc:1929] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:25.842011 28798 ts_tablet_manager.cc:1403] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.531s user 0.002s sys 0.016s
I20260618 13:59:25.843281 28801 log.cc:1199] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/3752f49c9fc64071a45e0ce4ff4cfef2
I20260618 13:59:25.846163 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5d046e0421142cd84c5b12d6a434f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.846735 28521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 957670718c8c4b8cb87115cc4a2fa861. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.852149 28801 ts_tablet_manager.cc:1950] T 3752f49c9fc64071a45e0ce4ff4cfef2 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:25.853508 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7aa9f65d8a5c46fdb745ee17af946dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.854368 28373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30f4bd3f86484caab48c1f0806299460. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.854710 28527 tablet_service.cc:1511] Processing CreateTablet for tablet 35d8dd2c388e43a0be380aa953ec2c82 (DEFAULT_TABLE table=default.Table_a8705c5cd49a40049ff1020a01aa7f5b [id=f9ff809bb5da4d048e7520cec3316f6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.855358 28510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad045effb32948b5b8e9be5bb7996577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.855298 28798 raft_consensus.cc:359] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.857306 28798 raft_consensus.cc:385] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:25.858222 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 650ed7ee8b134bc4b9f95028c54f99be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.849984 28659 tablet_service.cc:1511] Processing CreateTablet for tablet 41c92a594cd64982a8972e4a019d1cd8 (DEFAULT_TABLE table=default.Table_387aa217e7f04d2f826dc1f82c256a97 [id=d5294ec922b94ef7b1a4394f00462ff4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.848959 28661 tablet_service.cc:1511] Processing CreateTablet for tablet 9024952ab8094fcda1329562bbde8ef5 (DEFAULT_TABLE table=default.Table_55f1c253e54246eda98a7dcf2996952b [id=5e62ed351f73400ab201c1b5dd43f28f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.860126 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 316110bd2d024c7599fa2933d87e559e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.860183 28798 raft_consensus.cc:740] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.860436 28798 consensus_queue.cc:260] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:25.865592 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35d8dd2c388e43a0be380aa953ec2c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.868692 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3ba9186b3ff47d5a2b158fef619686e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.869486 28799 raft_consensus.cc:740] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:25.871948 28391 tablet_service.cc:1511] Processing CreateTablet for tablet 15483938a06e459789f87c6bb456bd15 (DEFAULT_TABLE table=default.Table_f37d21a9a59249d6a0b87c3788ff4873 [id=22eb9037fc66472793e9fd3f03fd7218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.875015 28372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9f7359c4a144ca89fe3e51d1365a425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.875595 28384 tablet_service.cc:1511] Processing CreateTablet for tablet fa24afe9db46408cace50f642f879f4b (DEFAULT_TABLE table=default.Table_4108c7ea79f5426395746346c44df521 [id=5a04afb74af64b17914f1587d7ddc542]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.868690 28517 tablet_service.cc:1511] Processing CreateTablet for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (DEFAULT_TABLE table=default.Table_7cd820b0428a41ada19a2f14560a67e4 [id=2c73a9d87a914f03b09ef422d49f7a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:25.880072 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 657e55fa53c447babfe91b1582806e6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.880247 28346 leader_election.cc:304] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:25.882485 28934 raft_consensus.cc:2749] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:25.883710 28380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca2ee746c5024cfc9b31c470f9616e7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.884049 28799 consensus_queue.cc:260] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:25.888052 28346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 187477us
I20260618 13:59:25.889022 28934 raft_consensus.cc:493] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:25.889170 28934 raft_consensus.cc:515] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:25.889183 28226 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_9bfb035b965742abaaf21413be078bf6" } modify_external_catalogs: true
I20260618 13:59:25.889364 28226 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_9bfb035b965742abaaf21413be078bf6" } modify_external_catalogs: true
I20260618 13:59:25.889894 28934 leader_election.cc:290] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:25.892089 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15483938a06e459789f87c6bb456bd15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.897270 28799 ts_tablet_manager.cc:1434] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.220s user 0.000s sys 0.003s
I20260618 13:59:25.897639 28799 tablet_bootstrap.cc:492] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:25.907012 28346 leader_election.cc:343] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:25.907478 28346 leader_election.cc:304] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:25.907878 28934 raft_consensus.cc:2749] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:25.919016 28799 tablet_bootstrap.cc:654] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:25.935962 28374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf5e04eb43ed4cf493c0f6f08e22baeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.953110 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet ac58e939730c4af681c2f08bcfbb6602 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:25.958477 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c2b924a096b479cad31c29bf7c99172" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:25.970132 28798 ts_tablet_manager.cc:1434] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.126s user 0.002s sys 0.002s
I20260618 13:59:25.971088 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 650ed7ee8b134bc4b9f95028c54f99be. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:25.981070 28346 leader_election.cc:343] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:25.987829 28375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35d8dd2c388e43a0be380aa953ec2c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.968952 28649 tablet_service.cc:1511] Processing CreateTablet for tablet f3ba9186b3ff47d5a2b158fef619686e (DEFAULT_TABLE table=default.Table_3ef2ff183a6044e2a9e069a4eac133b8 [id=60e91b2138bf41a5a8dee65c1874678a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.002334 28661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9024952ab8094fcda1329562bbde8ef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.004300 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 957670718c8c4b8cb87115cc4a2fa861. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.006616 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5d046e0421142cd84c5b12d6a434f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.007798 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41c92a594cd64982a8972e4a019d1cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.008437 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15483938a06e459789f87c6bb456bd15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.016541 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa24afe9db46408cace50f642f879f4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:25.946672 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 3752f49c9fc64071a45e0ce4ff4cfef2 (table default.Table_046616bed79749d5923cbd9ab9dff2b4 [id=af7cd415b54c4e4d8c7fb6f29dc8ff7b]) successfully deleted
W20260618 13:59:26.023844 28215 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 84952.6us
W20260618 13:59:26.025702 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet be59f988f0e34b9480f859cf88b9c223 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet be59f988f0e34b9480f859cf88b9c223 already in progress: creating tablet
I20260618 13:59:26.027920 28802 ts_tablet_manager.cc:1929] T c84189a4b1194c87bbdf81f6e0ed56e4 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:26.028170 28802 log.cc:1199] T c84189a4b1194c87bbdf81f6e0ed56e4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/c84189a4b1194c87bbdf81f6e0ed56e4
W20260618 13:59:26.027685 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet dd2385e6752d49429422fc854539e9c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd2385e6752d49429422fc854539e9c6 already in progress: creating tablet
I20260618 13:59:26.029596 28376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 957670718c8c4b8cb87115cc4a2fa861. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.031324 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c2b924a096b479cad31c29bf7c99172" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:26.033987 29182 raft_consensus.cc:493] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:26.034162 29182 raft_consensus.cc:515] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.034996 29182 leader_election.cc:290] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:26.035156 29182 raft_consensus.cc:493] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:26.035281 29182 raft_consensus.cc:515] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.035750 29182 leader_election.cc:290] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:26.037460 28390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9024952ab8094fcda1329562bbde8ef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.042546 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5d046e0421142cd84c5b12d6a434f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.045238 28378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 538077e459a44ed78c95f2e881b6e0b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.045845 28802 ts_tablet_manager.cc:1950] T c84189a4b1194c87bbdf81f6e0ed56e4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:26.046203 28377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3ba9186b3ff47d5a2b158fef619686e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.048259 29183 raft_consensus.cc:493] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:26.048527 29183 raft_consensus.cc:515] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:26.049587 29183 leader_election.cc:290] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:26.051869 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab50e1f17a74eee8cad942525002996" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:26.053100 28512 tablet_service.cc:1511] Processing CreateTablet for tablet ca2ee746c5024cfc9b31c470f9616e7d (DEFAULT_TABLE table=default.Table_72e5809d8cce4f3bbe8227efcb8e2b2c [id=dcd71daa60ee430d9ca46a1463d9a1bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.053213 28411 raft_consensus.cc:2468] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:26.056245 28517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf5e04eb43ed4cf493c0f6f08e22baeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.061288 28382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad045effb32948b5b8e9be5bb7996577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.065286 28511 tablet_service.cc:1511] Processing CreateTablet for tablet f9f7359c4a144ca89fe3e51d1365a425 (DEFAULT_TABLE table=default.Table_7d317fa28e3142a19d15c8360151c357 [id=bba3ab8821ca48a5aa1104094d78892a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.065881 28798 tablet_bootstrap.cc:492] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:26.065984 28515 tablet_service.cc:1511] Processing CreateTablet for tablet 30f4bd3f86484caab48c1f0806299460 (DEFAULT_TABLE table=default.Table_6dc717aafef14b22ab95830a6cf50642 [id=0751d5ab69704387856ad7d40431e74b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.066617 28512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca2ee746c5024cfc9b31c470f9616e7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.066371 28483 leader_election.cc:304] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2; no voters:
I20260618 13:59:26.069491 28802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59166 (request call id 337) took 1057 ms. Trace:
I20260618 13:59:26.073369 28802 rpcz_store.cc:276] 0618 13:59:24.993689 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:24.994285 (+ 596us) service_pool.cc:224] Handling call
0618 13:59:25.512025 (+517740us) ts_tablet_manager.cc:1151] Deleting tablet c84189a4b1194c87bbdf81f6e0ed56e4
0618 13:59:25.512035 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:25.769510 (+257475us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:26.027364 (+257854us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:26.051332 (+ 23968us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":517513}
I20260618 13:59:26.074082 28802 tablet_replica.cc:333] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:26.074445 28802 raft_consensus.cc:2243] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:26.074750 28802 raft_consensus.cc:2272] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:26.078481 28799 tablet_bootstrap.cc:492] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:26.079151 28799 ts_tablet_manager.cc:1403] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.181s user 0.003s sys 0.009s
I20260618 13:59:26.080839 28799 raft_consensus.cc:359] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:26.082784 28799 raft_consensus.cc:385] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:26.083026 28799 raft_consensus.cc:740] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:26.085417 28515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30f4bd3f86484caab48c1f0806299460. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.092268 28511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9f7359c4a144ca89fe3e51d1365a425. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:26.093003 28483 leader_election.cc:343] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 5df99b28e9ae4b6bbad0fe0dafd125d6
I20260618 13:59:26.100890 29182 raft_consensus.cc:2804] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:26.104158 28802 ts_tablet_manager.cc:1916] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:26.106662 29182 raft_consensus.cc:493] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260618 13:59:26.106925 28346 leader_election.cc:343] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:26.103399 28799 consensus_queue.cc:260] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:26.109478 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df99b28e9ae4b6bbad0fe0dafd125d6" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:26.109478 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516421cd843b44769efa2b555697c065" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:26.111029 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab50e1f17a74eee8cad942525002996" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:26.115311 29182 raft_consensus.cc:3060] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:26.115798 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516421cd843b44769efa2b555697c065" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:26.116376 28484 leader_election.cc:343] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:26.116523 28484 leader_election.cc:304] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:26.116966 28484 leader_election.cc:343] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:26.118130 28799 ts_tablet_manager.cc:1434] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.039s user 0.002s sys 0.002s
I20260618 13:59:26.119729 28649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3ba9186b3ff47d5a2b158fef619686e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.122776 29185 raft_consensus.cc:2749] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:26.124972 29182 raft_consensus.cc:515] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.125587 29182 leader_election.cc:290] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:26.125963 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab50e1f17a74eee8cad942525002996" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:26.126143 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab50e1f17a74eee8cad942525002996" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:26.122793 28527 tablet_service.cc:1511] Processing CreateTablet for tablet fa24afe9db46408cace50f642f879f4b (DEFAULT_TABLE table=default.Table_4108c7ea79f5426395746346c44df521 [id=5a04afb74af64b17914f1587d7ddc542]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:26.127117 28484 leader_election.cc:343] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:26.129074 28797 tablet_bootstrap.cc:492] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:26.129249 28797 ts_tablet_manager.cc:1403] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.557s user 0.034s sys 0.100s
I20260618 13:59:26.130119 28797 raft_consensus.cc:359] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.130288 28797 raft_consensus.cc:385] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:26.130371 28797 raft_consensus.cc:740] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:26.130681 28797 consensus_queue.cc:260] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.140354 28800 ts_tablet_manager.cc:1929] T bd6b0cdffd5a44b5b45b885f6e88eb9d P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260618 13:59:26.140447 28522 consensus_meta.cc:287] T f54910015987484e95b295352d859315 P 97859748af2c41879ba2a668dda999d2: Time spent flushing consensus metadata: real 0.504s user 0.005s sys 0.000s
I20260618 13:59:26.141532 28800 log.cc:1199] T bd6b0cdffd5a44b5b45b885f6e88eb9d P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/bd6b0cdffd5a44b5b45b885f6e88eb9d
I20260618 13:59:26.141887 28798 tablet_bootstrap.cc:654] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:26.142858 28800 ts_tablet_manager.cc:1950] T bd6b0cdffd5a44b5b45b885f6e88eb9d P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:26.142954 28516 tablet_service.cc:1558] Processing DeleteTablet for tablet 5df99b28e9ae4b6bbad0fe0dafd125d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:26.144491 28801 tablet_replica.cc:333] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:26.144709 28801 raft_consensus.cc:2243] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:26.144904 28801 raft_consensus.cc:2272] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:26.145457 28801 ts_tablet_manager.cc:1916] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:26.147313 28411 raft_consensus.cc:3060] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:26.151355 28411 raft_consensus.cc:2468] T bab50e1f17a74eee8cad942525002996 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:26.153115 28483 leader_election.cc:304] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2; no voters: d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:26.153975 29185 raft_consensus.cc:2804] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:26.154809 29185 raft_consensus.cc:697] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:26.160553 28799 tablet_bootstrap.cc:492] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:26.160792 28797 ts_tablet_manager.cc:1434] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.031s user 0.000s sys 0.011s
I20260618 13:59:26.161299 28797 tablet_bootstrap.cc:492] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:26.165618 28346 leader_election.cc:304] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:26.029364 28215 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 18d8b609b96a4e81b57f69bff522eeea because the tablet was not found. No further retry: Not found: Tablet not found: 18d8b609b96a4e81b57f69bff522eeea
W20260618 13:59:26.172937 28215 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 143672us
W20260618 13:59:26.173343 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet fe750fc5d998448caeab0a6c808dfba7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe750fc5d998448caeab0a6c808dfba7 already in progress: creating tablet
W20260618 13:59:26.173599 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 7d4144b1b5ea4e968672616bc22bead8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d4144b1b5ea4e968672616bc22bead8 already in progress: creating tablet
W20260618 13:59:26.173822 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet b6dd422c05324c989e29b20ae2afabad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6dd422c05324c989e29b20ae2afabad already in progress: creating tablet
W20260618 13:59:26.174042 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 0463fa2e30554abcbfc8f37b702e81d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0463fa2e30554abcbfc8f37b702e81d4 already in progress: creating tablet
W20260618 13:59:26.174253 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d667ea5f029f46d89fcb3be03069b97b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d667ea5f029f46d89fcb3be03069b97b already in progress: creating tablet
W20260618 13:59:26.174523 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet b0285c75c632457383e81805ab036aaf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0285c75c632457383e81805ab036aaf already in progress: creating tablet
I20260618 13:59:26.175295 28519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 538077e459a44ed78c95f2e881b6e0b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.175697 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet b1f734a3f29b42e591d5564e99292b5e (table default.Table_26e7d8c4f91144268a2a43e83992667d [id=803d5220ce1a4dcfaf27d93c574be2ac]) successfully deleted
I20260618 13:59:26.155115 29185 consensus_queue.cc:237] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:26.175948 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet fe750fc5d998448caeab0a6c808dfba7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe750fc5d998448caeab0a6c808dfba7 already in progress: creating tablet
I20260618 13:59:26.176695 29183 raft_consensus.cc:2749] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:26.185257 28797 tablet_bootstrap.cc:654] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:26.190805 28800 tablet_replica.cc:333] T b739fa1847184290833e13a2515f3a3a P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
W20260618 13:59:26.190963 28346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 76560.5us
I20260618 13:59:26.191509 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet c84189a4b1194c87bbdf81f6e0ed56e4 (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]) successfully deleted
I20260618 13:59:26.192613 29183 raft_consensus.cc:493] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:26.192766 29183 raft_consensus.cc:515] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:26.193848 29183 leader_election.cc:290] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:26.193835 28800 raft_consensus.cc:2243] T b739fa1847184290833e13a2515f3a3a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:26.195802 28800 raft_consensus.cc:2272] T b739fa1847184290833e13a2515f3a3a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:26.196503 28800 ts_tablet_manager.cc:1916] T b739fa1847184290833e13a2515f3a3a P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:26.202464 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa24afe9db46408cace50f642f879f4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.209354 28798 tablet_bootstrap.cc:492] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:26.209537 28798 ts_tablet_manager.cc:1403] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.170s user 0.003s sys 0.007s
I20260618 13:59:26.212396 28798 raft_consensus.cc:359] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:26.213160 28346 leader_election.cc:343] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:26.213479 28346 leader_election.cc:343] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:26.213626 28346 leader_election.cc:304] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:26.213894 29183 raft_consensus.cc:2749] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:26.214608 28798 raft_consensus.cc:385] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:26.214728 28798 raft_consensus.cc:740] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:26.215024 28798 consensus_queue.cc:260] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.215408 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f81428f804a485c967026c4527fd250" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:26.216128 28346 leader_election.cc:343] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:26.221711 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f81428f804a485c967026c4527fd250" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:26.227579 28797 tablet_bootstrap.cc:492] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:26.231434 28797 ts_tablet_manager.cc:1403] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.067s user 0.005s sys 0.006s
I20260618 13:59:26.232402 28797 raft_consensus.cc:359] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:26.232740 28797 raft_consensus.cc:385] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:26.232867 28797 raft_consensus.cc:740] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
W20260618 13:59:26.232928 28215 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.100.3:35503, user_credentials={real_user=slave}} blocked reactor thread for 56983us
W20260618 13:59:26.233232 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 18d8b609b96a4e81b57f69bff522eeea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 18d8b609b96a4e81b57f69bff522eeea already in progress: creating tablet
I20260618 13:59:26.233233 28797 consensus_queue.cc:260] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:26.233489 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 7d4144b1b5ea4e968672616bc22bead8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d4144b1b5ea4e968672616bc22bead8 already in progress: creating tablet
I20260618 13:59:26.234297 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet bd6b0cdffd5a44b5b45b885f6e88eb9d (table default.Table_f32d5142bcf64f5c9e94b015dd620e10 [id=cfb012a9137e481eb0e03145a6404788]) successfully deleted
I20260618 13:59:26.234041 28797 ts_tablet_manager.cc:1434] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 13:59:26.234694 29190 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7637285570574a6dbcef1de6f84ecac0)
I20260618 13:59:26.234867 29190 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:26.234787 28797 tablet_bootstrap.cc:492] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:26.241681 28797 tablet_bootstrap.cc:654] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:26.242595 28799 tablet_bootstrap.cc:654] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:26.247182 28346 leader_election.cc:343] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:26.247344 28346 leader_election.cc:304] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:26.248908 29183 raft_consensus.cc:2749] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:26.252571 28802 ts_tablet_manager.cc:1929] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:26.252789 28802 log.cc:1199] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/c552d1316e3340f4bcfa39618ae309b8
I20260618 13:59:26.253232 28802 ts_tablet_manager.cc:1950] T c552d1316e3340f4bcfa39618ae309b8 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:26.254170 28802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59166 (request call id 341) took 1091 ms. Trace:
W20260618 13:59:26.255960 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet b6dd422c05324c989e29b20ae2afabad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6dd422c05324c989e29b20ae2afabad already in progress: creating tablet
W20260618 13:59:26.256330 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 0463fa2e30554abcbfc8f37b702e81d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0463fa2e30554abcbfc8f37b702e81d4 already in progress: creating tablet
W20260618 13:59:26.256556 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet dd2385e6752d49429422fc854539e9c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd2385e6752d49429422fc854539e9c6 already in progress: creating tablet
W20260618 13:59:26.256842 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet b0285c75c632457383e81805ab036aaf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0285c75c632457383e81805ab036aaf already in progress: creating tablet
I20260618 13:59:26.259279 29190 leader_election.cc:290] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a950ab7cd66b4cb0b29592aa692c2dd0 (127.27.100.61:45967), 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083)
I20260618 13:59:26.263311 28798 ts_tablet_manager.cc:1434] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.021s user 0.001s sys 0.002s
I20260618 13:59:26.263790 28798 tablet_bootstrap.cc:492] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:26.265537 28798 tablet_bootstrap.cc:654] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:26.272451 28797 tablet_bootstrap.cc:492] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:26.272704 28797 ts_tablet_manager.cc:1403] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.038s user 0.001s sys 0.003s
I20260618 13:59:26.272881 28800 ts_tablet_manager.cc:1929] T b739fa1847184290833e13a2515f3a3a P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:26.273029 28800 log.cc:1199] T b739fa1847184290833e13a2515f3a3a P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/b739fa1847184290833e13a2515f3a3a
I20260618 13:59:26.273428 28800 ts_tablet_manager.cc:1950] T b739fa1847184290833e13a2515f3a3a P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:26.273753 28797 raft_consensus.cc:359] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.274013 28797 raft_consensus.cc:385] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:26.274616 28797 raft_consensus.cc:740] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:26.274909 28797 consensus_queue.cc:260] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.275665 28797 ts_tablet_manager.cc:1434] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260618 13:59:26.276013 28797 tablet_bootstrap.cc:492] T 3752f49c9fc64071a45e0ce4ff4cfef2 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:26.254299 28802 rpcz_store.cc:276] 0618 13:59:25.162811 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.162863 (+ 52us) service_pool.cc:224] Handling call
0618 13:59:26.074003 (+911140us) ts_tablet_manager.cc:1151] Deleting tablet c552d1316e3340f4bcfa39618ae309b8
0618 13:59:26.074017 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:26.182408 (+108391us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:26.225349 (+ 42941us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:26.254113 (+ 28764us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":910920}
I20260618 13:59:26.278975 28802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59166 (request call id 342) took 1103 ms. Trace:
I20260618 13:59:26.281126 28797 tablet_bootstrap.cc:654] T 3752f49c9fc64071a45e0ce4ff4cfef2 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:26.281750 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet c552d1316e3340f4bcfa39618ae309b8 (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]) successfully deleted
I20260618 13:59:26.281878 28220 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_6f3c7c1d308e4901887fc9b602b38e4f" } modify_external_catalogs: true
I20260618 13:59:26.282480 28802 rpcz_store.cc:276] 0618 13:59:25.175209 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.175263 (+ 54us) service_pool.cc:224] Handling call
0618 13:59:26.278778 (+1103515us) ts_tablet_manager.cc:1151] Deleting tablet e6de1293572f4e75af3417f281b9a142
0618 13:59:26.278791 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:26.278916 (+ 125us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1103248}
I20260618 13:59:26.282859 28802 tablet_replica.cc:333] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:26.283051 28802 raft_consensus.cc:2243] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:26.283244 28802 raft_consensus.cc:2272] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:26.289907 28220 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_6f3c7c1d308e4901887fc9b602b38e4f" } modify_external_catalogs: true
I20260618 13:59:26.293735 28229 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_5d0d3ddc1f4a4badbb853268cb2bba65" } modify_external_catalogs: true
I20260618 13:59:26.295687 28797 tablet_bootstrap.cc:492] T 3752f49c9fc64071a45e0ce4ff4cfef2 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:26.295928 28797 ts_tablet_manager.cc:1403] T 3752f49c9fc64071a45e0ce4ff4cfef2 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.001s
I20260618 13:59:26.296792 28797 raft_consensus.cc:359] T 3752f49c9fc64071a45e0ce4ff4cfef2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.297061 28797 raft_consensus.cc:385] T 3752f49c9fc64071a45e0ce4ff4cfef2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:26.297178 28797 raft_consensus.cc:740] T 3752f49c9fc64071a45e0ce4ff4cfef2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:26.297523 28797 consensus_queue.cc:260] T 3752f49c9fc64071a45e0ce4ff4cfef2 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.298202 28797 ts_tablet_manager.cc:1434] T 3752f49c9fc64071a45e0ce4ff4cfef2 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:26.299151 28797 tablet_bootstrap.cc:492] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:26.299860 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet b739fa1847184290833e13a2515f3a3a (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]) successfully deleted
I20260618 13:59:26.293592 28239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 451 } } ignore_live_leader: false dest_uuid: "7637285570574a6dbcef1de6f84ecac0" is_pre_election: true
I20260618 13:59:26.301012 28655 tablet_service.cc:1511] Processing CreateTablet for tablet 9fa1a27fc9304e18b96cad9f923779f2 (DEFAULT_TABLE table=default.Table_aeeec1571f064ab1816cbf26635190fd [id=e408f407250f4c0eb56e1abe07fb6b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.301106 28658 tablet_service.cc:1511] Processing CreateTablet for tablet ba53ad1988064049a61fcca3a85f8779 (DEFAULT_TABLE table=default.Table_2a3bcaddd3034d67a5ea7c78f0ea9e7c [id=6c30b030a60b47e2be48dfe6e9cfdfdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.301414 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fa1a27fc9304e18b96cad9f923779f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.306335 28651 tablet_service.cc:1511] Processing CreateTablet for tablet 5e25027f423144b088dad955c7282921 (DEFAULT_TABLE table=default.Table_a7ff69763fdb4ff3b26b180bf05907d9 [id=e39e2c5718794a748361d7b453c65404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.306861 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e25027f423144b088dad955c7282921. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.310756 28658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba53ad1988064049a61fcca3a85f8779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.311919 28797 tablet_bootstrap.cc:654] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:26.322650 28657 tablet_service.cc:1511] Processing CreateTablet for tablet e5f7eaa7d218440ca2c35340c5c87814 (DEFAULT_TABLE table=default.Table_9a78338c3e414b0abea1281082202f57 [id=5697846ccd9c4c659755a0f9b8a66b72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.323132 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5f7eaa7d218440ca2c35340c5c87814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.333072 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_84c59c9ab3f54fb4930771088e194560"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:26.333791 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_84c59c9ab3f54fb4930771088e194560 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:26.335469 28797 tablet_bootstrap.cc:492] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:26.335762 28797 ts_tablet_manager.cc:1403] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.037s user 0.003s sys 0.004s
W20260618 13:59:26.335943 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet e6de1293572f4e75af3417f281b9a142 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e6de1293572f4e75af3417f281b9a142 already in progress: creating tablet
W20260618 13:59:26.336134 28214 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 46484.4us
I20260618 13:59:26.337040 28797 raft_consensus.cc:359] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.337251 28797 raft_consensus.cc:385] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:26.337343 28797 raft_consensus.cc:740] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:26.337615 28797 consensus_queue.cc:260] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.338258 28797 ts_tablet_manager.cc:1434] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:26.338621 28797 tablet_bootstrap.cc:492] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:26.340330 28797 tablet_bootstrap.cc:654] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:26.336874 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_19cfaeff1d174399b77939349481eb5a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:26.343916 28214 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet dc36fbea853a4d4e9f61909200239aed (table default.Table_dbc6162c06f74e71b792bd1f7b2388fe [id=49576dabdecf41aca1b15ff3fc0c113c], current schema version=6) failed: Not found: failed to reset TS proxy: No leader found: Tablet dc36fbea853a4d4e9f61909200239aed consensus state has no leader
I20260618 13:59:26.344851 28227 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 69d5e75b4a8247b29d69d07fad5ca5db (table default.Table_76724583c0db40378d257d673a601220 [id=5278d8eb60e4448ebae7afc3edb0db8e]). Expected version 1 got 0
I20260618 13:59:26.345887 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_7e5d58ad3e8043919b66e8890819535c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:26.347960 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e5d58ad3e8043919b66e8890819535c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:26.350308 28225 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_0a074f8e7c5443959918f5effc28e3d7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:26.350852 28225 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a074f8e7c5443959918f5effc28e3d7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:26.342751 28224 catalog_manager.cc:5696] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97859748af2c41879ba2a668dda999d2 (127.27.100.2). New cstate: current_term: 1 leader_uuid: "97859748af2c41879ba2a668dda999d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:26.354059 28797 tablet_bootstrap.cc:492] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
W20260618 13:59:26.354251 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_19cfaeff1d174399b77939349481eb5a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:26.354277 28797 ts_tablet_manager.cc:1403] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.000s
I20260618 13:59:26.355072 28797 raft_consensus.cc:359] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.355274 28797 raft_consensus.cc:385] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:26.355336 28797 raft_consensus.cc:740] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:26.355583 28797 consensus_queue.cc:260] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.356196 28797 ts_tablet_manager.cc:1434] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:26.356566 28797 tablet_bootstrap.cc:492] T 08a2dee3ac5a4bec96445a6001717eda P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:26.358201 28797 tablet_bootstrap.cc:654] T 08a2dee3ac5a4bec96445a6001717eda P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:26.359851 28229 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_5d0d3ddc1f4a4badbb853268cb2bba65" } modify_external_catalogs: true
I20260618 13:59:26.362102 28222 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_ff3779f2af2443428f9c1e3772091659" } new_table_name: "default.Table_af28c507a4b04f7097bd6a034351abc8" modify_external_catalogs: true
I20260618 13:59:26.362468 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 5df99b28e9ae4b6bbad0fe0dafd125d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:20 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:26.366714 28797 tablet_bootstrap.cc:492] T 08a2dee3ac5a4bec96445a6001717eda P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:26.366964 28797 ts_tablet_manager.cc:1403] T 08a2dee3ac5a4bec96445a6001717eda P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260618 13:59:26.367800 28797 raft_consensus.cc:359] T 08a2dee3ac5a4bec96445a6001717eda P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.368058 28797 raft_consensus.cc:385] T 08a2dee3ac5a4bec96445a6001717eda P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:26.368186 28797 raft_consensus.cc:740] T 08a2dee3ac5a4bec96445a6001717eda P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:26.368455 28797 consensus_queue.cc:260] T 08a2dee3ac5a4bec96445a6001717eda P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.369083 28797 ts_tablet_manager.cc:1434] T 08a2dee3ac5a4bec96445a6001717eda P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:26.369419 28797 tablet_bootstrap.cc:492] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:26.371107 28797 tablet_bootstrap.cc:654] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:26.376420 28655 tablet_service.cc:1511] Processing CreateTablet for tablet d2d54a901e9f4d5c89700979ce0618bf (DEFAULT_TABLE table=default.Table_0ec7ef7f4c6e48d5b196e45f78a4fbf2 [id=3b33c0ca00d248298402d6bcec792be4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.376964 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2d54a901e9f4d5c89700979ce0618bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.377647 28797 tablet_bootstrap.cc:492] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:26.377817 28797 ts_tablet_manager.cc:1403] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.004s
I20260618 13:59:26.378748 28797 raft_consensus.cc:359] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.379035 28797 raft_consensus.cc:385] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:26.379119 28797 raft_consensus.cc:740] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:26.379410 28797 consensus_queue.cc:260] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.379948 28797 ts_tablet_manager.cc:1434] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:26.380112 28656 tablet_service.cc:1511] Processing CreateTablet for tablet 80e99a85e83c46afa367e59e5ddb7dc7 (DEFAULT_TABLE table=default.Table_334d2fffc20b4a5688939d1067192640 [id=ca499a8d88084635a1bc0c1107ce4b86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.380304 28797 tablet_bootstrap.cc:492] T bab50e1f17a74eee8cad942525002996 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:26.380620 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80e99a85e83c46afa367e59e5ddb7dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.382136 28797 tablet_bootstrap.cc:654] T bab50e1f17a74eee8cad942525002996 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:26.382866 28652 tablet_service.cc:1511] Processing CreateTablet for tablet 2d1073b726ef4cbeac6c874fc8863e90 (DEFAULT_TABLE table=default.Table_bbf208e6e39a47349ab50198f178974e [id=8e4cf344c21640728c419a084f14c7b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.383286 28652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d1073b726ef4cbeac6c874fc8863e90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.386794 28647 tablet_service.cc:1511] Processing CreateTablet for tablet 7f83bb8c3c504d5f9affbaa8be3f1508 (DEFAULT_TABLE table=default.Table_a44222ec49e54499bbd3d2c256abc71c [id=3f176321fcf44af2b5a4103634b1908f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.387193 28647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f83bb8c3c504d5f9affbaa8be3f1508. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.391665 28661 tablet_service.cc:1511] Processing CreateTablet for tablet 054a7afb3b644aa49df78302cf8f02c9 (DEFAULT_TABLE table=default.Table_2f8e4182bb594c2bad77994c12ce821f [id=c70ab3da89b74f9ba473123d260c0f14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.392186 28661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054a7afb3b644aa49df78302cf8f02c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.399924 28649 tablet_service.cc:1558] Processing DeleteTablet for tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 with delete_type TABLET_DATA_DELETED (replaced by tablet ac58e939730c4af681c2f08bcfbb6602) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:26.403993 28658 tablet_service.cc:1511] Processing CreateTablet for tablet 438663f365e34fab87e34f434920cecf (DEFAULT_TABLE table=default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.404556 28658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 438663f365e34fab87e34f434920cecf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.406909 28663 tablet_service.cc:1558] Processing DeleteTablet for tablet 668c6c6f2034484ca5ccdd51c22dea1d with delete_type TABLET_DATA_DELETED (replaced by tablet 9da90366cb7442bb9f4bf3bab07db070) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:26.409834 28802 ts_tablet_manager.cc:1916] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:26.410318 28934 raft_consensus.cc:493] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:26.410483 28934 raft_consensus.cc:515] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:26.411199 28934 leader_election.cc:290] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:26.416482 28659 tablet_service.cc:1511] Processing CreateTablet for tablet b509f940a02545019c526e45e8372c97 (DEFAULT_TABLE table=default.Table_06cd6ce4fc4e4dcfb2349c87e228c7de [id=840c933184144aebb3659d939f679f00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.418478 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b509f940a02545019c526e45e8372c97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.418813 29183 raft_consensus.cc:493] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:26.427436 28377 tablet_service.cc:1511] Processing CreateTablet for tablet 054a7afb3b644aa49df78302cf8f02c9 (DEFAULT_TABLE table=default.Table_2f8e4182bb594c2bad77994c12ce821f [id=c70ab3da89b74f9ba473123d260c0f14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.428195 28377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054a7afb3b644aa49df78302cf8f02c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.430464 29183 raft_consensus.cc:515] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:26.431115 29183 leader_election.cc:290] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:26.431840 28382 tablet_service.cc:1511] Processing CreateTablet for tablet e5f7eaa7d218440ca2c35340c5c87814 (DEFAULT_TABLE table=default.Table_9a78338c3e414b0abea1281082202f57 [id=5697846ccd9c4c659755a0f9b8a66b72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.432224 28382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5f7eaa7d218440ca2c35340c5c87814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.433203 28379 tablet_service.cc:1511] Processing CreateTablet for tablet b509f940a02545019c526e45e8372c97 (DEFAULT_TABLE table=default.Table_06cd6ce4fc4e4dcfb2349c87e228c7de [id=840c933184144aebb3659d939f679f00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.434090 28379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b509f940a02545019c526e45e8372c97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.434064 28649 tablet_service.cc:1511] Processing CreateTablet for tablet aa93f065b4d64e7b85e30f0690fae541 (DEFAULT_TABLE table=default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.435791 28934 raft_consensus.cc:493] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:26.435951 28934 raft_consensus.cc:515] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.436537 28934 leader_election.cc:290] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:26.437395 28390 tablet_service.cc:1511] Processing CreateTablet for tablet 5e25027f423144b088dad955c7282921 (DEFAULT_TABLE table=default.Table_a7ff69763fdb4ff3b26b180bf05907d9 [id=e39e2c5718794a748361d7b453c65404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.437896 28390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e25027f423144b088dad955c7282921. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.443116 28376 tablet_service.cc:1511] Processing CreateTablet for tablet 9fa1a27fc9304e18b96cad9f923779f2 (DEFAULT_TABLE table=default.Table_aeeec1571f064ab1816cbf26635190fd [id=e408f407250f4c0eb56e1abe07fb6b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.443462 28376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fa1a27fc9304e18b96cad9f923779f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.445171 28226 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 827cc0987bd541759e334fb620f04ed5
I20260618 13:59:26.445199 28649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa93f065b4d64e7b85e30f0690fae541. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.446848 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet 827cc0987bd541759e334fb620f04ed5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:26.447816 28223 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet da4d4f03911f4ac099ecc2fa1d8af494
I20260618 13:59:26.447835 28378 tablet_service.cc:1511] Processing CreateTablet for tablet ba53ad1988064049a61fcca3a85f8779 (DEFAULT_TABLE table=default.Table_2a3bcaddd3034d67a5ea7c78f0ea9e7c [id=6c30b030a60b47e2be48dfe6e9cfdfdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.448165 28378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba53ad1988064049a61fcca3a85f8779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.450284 28220 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 02b5cc08ca0c4e719e0414b345211ba5
I20260618 13:59:26.450901 28384 tablet_service.cc:1511] Processing CreateTablet for tablet d2d54a901e9f4d5c89700979ce0618bf (DEFAULT_TABLE table=default.Table_0ec7ef7f4c6e48d5b196e45f78a4fbf2 [id=3b33c0ca00d248298402d6bcec792be4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.451261 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2d54a901e9f4d5c89700979ce0618bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.451427 28655 tablet_service.cc:1511] Processing CreateTablet for tablet 340f6bbf18eb412ca3afec237ff7752d (DEFAULT_TABLE table=default.Table_3eaa556cd32f4cc1a275c041a0ca104b [id=e6529022f4a2497ea0ed450a84825b93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.454836 28647 tablet_service.cc:1511] Processing CreateTablet for tablet 55ae29c3b7ed44cfa659130745037aa6 (DEFAULT_TABLE table=default.Table_5d485acde41448d984aa79fa58b1866b [id=726324d365b94ff6bac24030222ca34f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.455525 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 80e99a85e83c46afa367e59e5ddb7dc7 (DEFAULT_TABLE table=default.Table_334d2fffc20b4a5688939d1067192640 [id=ca499a8d88084635a1bc0c1107ce4b86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.455909 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80e99a85e83c46afa367e59e5ddb7dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.457579 28375 tablet_service.cc:1511] Processing CreateTablet for tablet 7f83bb8c3c504d5f9affbaa8be3f1508 (DEFAULT_TABLE table=default.Table_a44222ec49e54499bbd3d2c256abc71c [id=3f176321fcf44af2b5a4103634b1908f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.457906 28375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f83bb8c3c504d5f9affbaa8be3f1508. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.459661 28374 tablet_service.cc:1511] Processing CreateTablet for tablet 2d1073b726ef4cbeac6c874fc8863e90 (DEFAULT_TABLE table=default.Table_bbf208e6e39a47349ab50198f178974e [id=8e4cf344c21640728c419a084f14c7b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.459985 28374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d1073b726ef4cbeac6c874fc8863e90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.461352 28229 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 411b0c333ae7450f9ada90afee553615
I20260618 13:59:26.461746 28380 tablet_service.cc:1511] Processing CreateTablet for tablet aa93f065b4d64e7b85e30f0690fae541 (DEFAULT_TABLE table=default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.462119 28380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa93f065b4d64e7b85e30f0690fae541. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.463881 28372 tablet_service.cc:1511] Processing CreateTablet for tablet 438663f365e34fab87e34f434920cecf (DEFAULT_TABLE table=default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.464218 28372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 438663f365e34fab87e34f434920cecf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:26.464339 28222 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fe1139c409e84070a2f496a85545e8d5 consensus state has no leader
I20260618 13:59:26.464644 28222 master_service.cc:960] ReplaceTablet: received request to replace tablet 591465e2b57247a2b224cc8b09225811 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:26.465561 28656 tablet_service.cc:1511] Processing CreateTablet for tablet fe1139c409e84070a2f496a85545e8d5 (DEFAULT_TABLE table=default.Table_ff3779f2af2443428f9c1e3772091659 [id=c3a54561a2e7448ebb47b0234579944e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.465989 28385 tablet_service.cc:1511] Processing CreateTablet for tablet fe1139c409e84070a2f496a85545e8d5 (DEFAULT_TABLE table=default.Table_ff3779f2af2443428f9c1e3772091659 [id=c3a54561a2e7448ebb47b0234579944e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.466063 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d5e75b4a8247b29d69d07fad5ca5db" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:26.466332 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe1139c409e84070a2f496a85545e8d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.467278 28663 tablet_service.cc:1511] Processing CreateTablet for tablet fc1968e5ce2e42a5b836c90b0904f158 (DEFAULT_TABLE table=default.Table_dfe94c1f858644c98dba7db12d019477 [id=ea67f187738d41c993153f788e016e73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.468243 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet 827cc0987bd541759e334fb620f04ed5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:26.468472 28373 tablet_service.cc:1558] Processing DeleteTablet for tablet da4d4f03911f4ac099ecc2fa1d8af494 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:26.468819 28389 tablet_service.cc:1511] Processing CreateTablet for tablet 340f6bbf18eb412ca3afec237ff7752d (DEFAULT_TABLE table=default.Table_3eaa556cd32f4cc1a275c041a0ca104b [id=e6529022f4a2497ea0ed450a84825b93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.469117 28389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 340f6bbf18eb412ca3afec237ff7752d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.469254 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac58e939730c4af681c2f08bcfbb6602" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:26.470520 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 340f6bbf18eb412ca3afec237ff7752d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.470530 28387 tablet_service.cc:1558] Processing DeleteTablet for tablet 02b5cc08ca0c4e719e0414b345211ba5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:26.470867 28381 tablet_service.cc:1511] Processing CreateTablet for tablet 55ae29c3b7ed44cfa659130745037aa6 (DEFAULT_TABLE table=default.Table_5d485acde41448d984aa79fa58b1866b [id=726324d365b94ff6bac24030222ca34f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.470917 28651 tablet_service.cc:1558] Processing DeleteTablet for tablet 411b0c333ae7450f9ada90afee553615 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:26.471167 28381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55ae29c3b7ed44cfa659130745037aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.471720 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a0691de253b4b008e5d995fd6b876c2" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:26.471828 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d667ea5f029f46d89fcb3be03069b97b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d667ea5f029f46d89fcb3be03069b97b already in progress: creating tablet
I20260618 13:59:26.472294 28662 tablet_service.cc:1511] Processing CreateTablet for tablet 97a9f949999446f09359629fe483bf67 (DEFAULT_TABLE table=default.Table_9424eec16a284672a0fe7ab3d0852b40 [id=070ea48af78346aa8ff43f2bb369730c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.472788 28654 tablet_service.cc:1511] Processing CreateTablet for tablet c38a2156a55e4e5eb91d101e9a61eecd (DEFAULT_TABLE table=default.Table_dc44ae6b83d844b6a16ce10ba6dce0ad [id=2e7a199c805d4453b2137b5fa69d0b4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.472586 28800 tablet_replica.cc:333] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:26.473609 28647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55ae29c3b7ed44cfa659130745037aa6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:26.474639 28346 leader_election.cc:343] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:26.474877 28346 leader_election.cc:343] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:26.474912 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97a9f949999446f09359629fe483bf67. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:26.475046 28346 leader_election.cc:343] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:26.475492 28387 tablet_service.cc:1511] Processing CreateTablet for tablet fc1968e5ce2e42a5b836c90b0904f158 (DEFAULT_TABLE table=default.Table_dfe94c1f858644c98dba7db12d019477 [id=ea67f187738d41c993153f788e016e73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.475901 28387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc1968e5ce2e42a5b836c90b0904f158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.476578 28373 tablet_service.cc:1558] Processing DeleteTablet for tablet 411b0c333ae7450f9ada90afee553615 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:26.477006 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_c35470a550534fa2ad072f932e981df5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:26.477366 28373 tablet_service.cc:1511] Processing CreateTablet for tablet c38a2156a55e4e5eb91d101e9a61eecd (DEFAULT_TABLE table=default.Table_dc44ae6b83d844b6a16ce10ba6dce0ad [id=2e7a199c805d4453b2137b5fa69d0b4e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:26.477566 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c35470a550534fa2ad072f932e981df5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:26.477667 28373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c38a2156a55e4e5eb91d101e9a61eecd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.478364 28391 tablet_service.cc:1511] Processing CreateTablet for tablet 97a9f949999446f09359629fe483bf67 (DEFAULT_TABLE table=default.Table_9424eec16a284672a0fe7ab3d0852b40 [id=070ea48af78346aa8ff43f2bb369730c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.478678 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97a9f949999446f09359629fe483bf67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.479398 28223 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_b9f02faebbe34f9299f751f0e9f17d57" } new_table_name: "default.Table_97b6bb39b3bf4612be391b9dd78d1646" modify_external_catalogs: true
I20260618 13:59:26.482776 28228 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_644a49408bd64f4d98eaa6650b3b93e3" } new_table_name: "default.Table_69a1929cd67c4491bd769e49f6d4ab09" modify_external_catalogs: true
I20260618 13:59:26.483577 28226 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 69d5e75b4a8247b29d69d07fad5ca5db (table default.Table_76724583c0db40378d257d673a601220 [id=5278d8eb60e4448ebae7afc3edb0db8e]). Expected version 1 got 0
I20260618 13:59:26.483999 28225 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_697444611a2643bfb1fe30fc545c47f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:26.484351 28226 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_55f1c253e54246eda98a7dcf2996952b" } new_table_name: "default.Table_d334a203a2784021b5e7998a87a91a50" modify_external_catalogs: true
W20260618 13:59:26.484544 28225 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_697444611a2643bfb1fe30fc545c47f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:26.486399 28224 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_1333d91d141a46acac33460026215195" } modify_external_catalogs: true
I20260618 13:59:26.486604 28224 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_1333d91d141a46acac33460026215195" } modify_external_catalogs: true
I20260618 13:59:26.488243 28229 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_06cd6ce4fc4e4dcfb2349c87e228c7de" } new_table_name: "default.Table_12d6cc95071949c6a7a297cd08b5f898" modify_external_catalogs: true
I20260618 13:59:26.489449 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_f7b8e1561d67469b9a3709475d661de9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:26.489939 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f7b8e1561d67469b9a3709475d661de9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:26.491842 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_9a10a9aa97574e7c89586d1e4e863171"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:26.492436 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9a10a9aa97574e7c89586d1e4e863171 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:26.496742 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet da4d4f03911f4ac099ecc2fa1d8af494 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:26.509902 28802 ts_tablet_manager.cc:1929] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:26.510069 28802 log.cc:1199] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/2188ebf1727240cd886ba7662f8fcf43
I20260618 13:59:26.510538 28802 ts_tablet_manager.cc:1950] T 2188ebf1727240cd886ba7662f8fcf43 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:26.516037 28652 tablet_service.cc:1511] Processing CreateTablet for tablet 7766014ee9814f01ae7ac3e849ca3d50 (DEFAULT_TABLE table=default.Table_c528008fd94b4fc8b7341f5b23dff76b [id=516d3679e0e84ab28429e59b6895db4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.517004 28651 tablet_service.cc:1558] Processing DeleteTablet for tablet 3752f49c9fc64071a45e0ce4ff4cfef2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:26.519065 28388 tablet_service.cc:1511] Processing CreateTablet for tablet 7766014ee9814f01ae7ac3e849ca3d50 (DEFAULT_TABLE table=default.Table_c528008fd94b4fc8b7341f5b23dff76b [id=516d3679e0e84ab28429e59b6895db4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.519552 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7766014ee9814f01ae7ac3e849ca3d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.521273 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc1968e5ce2e42a5b836c90b0904f158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.522579 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 02b5cc08ca0c4e719e0414b345211ba5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:26.523445 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c38a2156a55e4e5eb91d101e9a61eecd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.523494 28386 tablet_service.cc:1511] Processing CreateTablet for tablet 3a783590159f4a80964fbbcb4a24eef9 (DEFAULT_TABLE table=default.Table_abe180e115a04052ae89cc92d3c1c1e6 [id=4cbcd521f63047a38f5a9b9def56df7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.523947 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a783590159f4a80964fbbcb4a24eef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.524047 28652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7766014ee9814f01ae7ac3e849ca3d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.528158 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet dca54111fea047d9ae0fabe376d89382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:26.531440 28653 tablet_service.cc:1511] Processing CreateTablet for tablet b968a8d4014341ca9c5964263721ac79 (DEFAULT_TABLE table=default.Table_b8741a7fd29f4e78a948b1c419595a85 [id=6f4d908593ae40f09e2c94cef218e1fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.536407 28651 tablet_service.cc:1558] Processing DeleteTablet for tablet 95db5ddaf5d5486988750efed5972eb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:26.537068 28657 tablet_service.cc:1511] Processing CreateTablet for tablet 3a783590159f4a80964fbbcb4a24eef9 (DEFAULT_TABLE table=default.Table_abe180e115a04052ae89cc92d3c1c1e6 [id=4cbcd521f63047a38f5a9b9def56df7f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:26.542272 28223 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 15180aefba0544329daee74308ef5010 consensus state has no leader
I20260618 13:59:26.542598 28223 master_service.cc:960] ReplaceTablet: received request to replace tablet b51d0851efb34eaa9e9e4be6b292ef62 from {username='slave'} at 127.0.0.1:45138
W20260618 13:59:26.546537 28226 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9024952ab8094fcda1329562bbde8ef5 consensus state has no leader
I20260618 13:59:26.546988 28224 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 8b88ed7cf79c4929a8704f66691cbed4
W20260618 13:59:26.549870 28228 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1d8b3cfac0f648cd8b1c51102132c676 consensus state has no leader
I20260618 13:59:26.550019 29186 raft_consensus.cc:493] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:26.550211 29186 raft_consensus.cc:515] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.551278 29186 leader_election.cc:290] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:26.551865 28651 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b88ed7cf79c4929a8704f66691cbed4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43976
W20260618 13:59:26.552093 28617 leader_election.cc:343] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 668c6c6f2034484ca5ccdd51c22dea1d
I20260618 13:59:26.552853 29186 raft_consensus.cc:493] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:26.553054 29186 raft_consensus.cc:515] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.553825 29186 leader_election.cc:290] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:26.554620 28617 leader_election.cc:343] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: dca54111fea047d9ae0fabe376d89382
I20260618 13:59:26.554776 28651 tablet_service.cc:1558] Processing DeleteTablet for tablet 08a2dee3ac5a4bec96445a6001717eda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:26.554955 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet b0c9d4131ea24809ac081e7a73a21e1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:26.555598 28225 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_8c9c9a6eea174bd68d8fb9d9ebaaf603"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:26.555594 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_38a3b325a9514a3d829c6ba288f9673a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:26.556250 28225 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8c9c9a6eea174bd68d8fb9d9ebaaf603 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:26.556478 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_38a3b325a9514a3d829c6ba288f9673a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:26.557492 28379 tablet_service.cc:1511] Processing CreateTablet for tablet b968a8d4014341ca9c5964263721ac79 (DEFAULT_TABLE table=default.Table_b8741a7fd29f4e78a948b1c419595a85 [id=6f4d908593ae40f09e2c94cef218e1fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.558339 28379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b968a8d4014341ca9c5964263721ac79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.562507 28378 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b88ed7cf79c4929a8704f66691cbed4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:26.563465 28802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59166 (request call id 344) took 1388 ms. Trace:
I20260618 13:59:26.563627 28802 rpcz_store.cc:276] 0618 13:59:25.175445 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.175650 (+ 205us) service_pool.cc:224] Handling call
0618 13:59:26.282774 (+1107124us) ts_tablet_manager.cc:1151] Deleting tablet 2188ebf1727240cd886ba7662f8fcf43
0618 13:59:26.282784 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:26.420549 (+137765us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:26.482712 (+ 62163us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:26.563425 (+ 80713us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1106913}
I20260618 13:59:26.564038 28802 tablet_replica.cc:333] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:26.564332 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 2188ebf1727240cd886ba7662f8fcf43 (table default.Table_fac3fdd9b466497683bc00300a85fed1 [id=a4ac38531af34b4681abba96b42ffb7f]) successfully deleted
I20260618 13:59:26.564461 28802 raft_consensus.cc:2243] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:26.564615 28802 raft_consensus.cc:2272] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:26.565162 28802 ts_tablet_manager.cc:1916] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:26.566753 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe1139c409e84070a2f496a85545e8d5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:26.567768 28229 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b509f940a02545019c526e45e8372c97 consensus state has no leader
I20260618 13:59:26.589332 28222 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 591465e2b57247a2b224cc8b09225811
I20260618 13:59:26.589851 28222 catalog_manager.cc:6784] ReplaceTablet: tablet 591465e2b57247a2b224cc8b09225811 deleted and replaced by tablet d4667fe48f0f4eff9c6c8baabdab58d4
I20260618 13:59:26.590961 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet 591465e2b57247a2b224cc8b09225811 with delete_type TABLET_DATA_DELETED (replaced by tablet d4667fe48f0f4eff9c6c8baabdab58d4) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:26.594702 28800 raft_consensus.cc:2243] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:26.595000 28800 raft_consensus.cc:2272] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:26.597649 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dca54111fea047d9ae0fabe376d89382" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:26.601387 29185 raft_consensus.cc:493] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:26.601559 29185 raft_consensus.cc:515] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.603160 28520 tablet_service.cc:1511] Processing CreateTablet for tablet b509f940a02545019c526e45e8372c97 (DEFAULT_TABLE table=default.Table_06cd6ce4fc4e4dcfb2349c87e228c7de [id=840c933184144aebb3659d939f679f00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.603706 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a783590159f4a80964fbbcb4a24eef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.604811 28517 tablet_service.cc:1511] Processing CreateTablet for tablet 054a7afb3b644aa49df78302cf8f02c9 (DEFAULT_TABLE table=default.Table_2f8e4182bb594c2bad77994c12ce821f [id=c70ab3da89b74f9ba473123d260c0f14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.606771 28801 ts_tablet_manager.cc:1929] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:26.606930 28801 log.cc:1199] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/5df99b28e9ae4b6bbad0fe0dafd125d6
I20260618 13:59:26.609817 28519 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:34092 (request call id 356) took 1007 ms. Trace:
I20260618 13:59:26.610471 28800 ts_tablet_manager.cc:1916] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:26.610677 28519 rpcz_store.cc:276] 0618 13:59:25.602299 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.602493 (+ 194us) service_pool.cc:224] Handling call
0618 13:59:25.614501 (+ 12008us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0618 13:59:25.614524 (+ 23us) ts_tablet_manager.cc:682] Creating new metadata...
0618 13:59:26.229751 (+615227us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:26.609674 (+379923us) tablet_replica.cc:206] Creating consensus instance
0618 13:59:26.609784 (+ 110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260618 13:59:26.611640 28510 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:34092 (request call id 355) took 1004 ms. Trace:
I20260618 13:59:26.614074 28514 tablet_service.cc:1511] Processing CreateTablet for tablet 5e25027f423144b088dad955c7282921 (DEFAULT_TABLE table=default.Table_a7ff69763fdb4ff3b26b180bf05907d9 [id=e39e2c5718794a748361d7b453c65404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.614406 28510 rpcz_store.cc:276] 0618 13:59:25.602203 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.602269 (+ 66us) service_pool.cc:224] Handling call
0618 13:59:25.615017 (+ 12748us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0618 13:59:25.615055 (+ 38us) ts_tablet_manager.cc:682] Creating new metadata...
0618 13:59:26.191492 (+576437us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:26.601223 (+409731us) tablet_replica.cc:206] Creating consensus instance
0618 13:59:26.606350 (+ 5127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260618 13:59:26.615574 28510 tablet_service.cc:1511] Processing CreateTablet for tablet 9fa1a27fc9304e18b96cad9f923779f2 (DEFAULT_TABLE table=default.Table_aeeec1571f064ab1816cbf26635190fd [id=e408f407250f4c0eb56e1abe07fb6b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.615825 28512 tablet_service.cc:1511] Processing CreateTablet for tablet d2d54a901e9f4d5c89700979ce0618bf (DEFAULT_TABLE table=default.Table_0ec7ef7f4c6e48d5b196e45f78a4fbf2 [id=3b33c0ca00d248298402d6bcec792be4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.616554 28801 ts_tablet_manager.cc:1950] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:26.614224 29185 leader_election.cc:290] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:26.618186 28528 tablet_service.cc:1511] Processing CreateTablet for tablet aa93f065b4d64e7b85e30f0690fae541 (DEFAULT_TABLE table=default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.618345 28518 tablet_service.cc:1511] Processing CreateTablet for tablet fe1139c409e84070a2f496a85545e8d5 (DEFAULT_TABLE table=default.Table_ff3779f2af2443428f9c1e3772091659 [id=c3a54561a2e7448ebb47b0234579944e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.618499 28525 tablet_service.cc:1511] Processing CreateTablet for tablet 80e99a85e83c46afa367e59e5ddb7dc7 (DEFAULT_TABLE table=default.Table_334d2fffc20b4a5688939d1067192640 [id=ca499a8d88084635a1bc0c1107ce4b86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.620764 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 340f6bbf18eb412ca3afec237ff7752d (DEFAULT_TABLE table=default.Table_3eaa556cd32f4cc1a275c041a0ca104b [id=e6529022f4a2497ea0ed450a84825b93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.622170 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet da4d4f03911f4ac099ecc2fa1d8af494 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:26.622438 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet 827cc0987bd541759e334fb620f04ed5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:26.622586 28524 tablet_service.cc:1511] Processing CreateTablet for tablet 7f83bb8c3c504d5f9affbaa8be3f1508 (DEFAULT_TABLE table=default.Table_a44222ec49e54499bbd3d2c256abc71c [id=3f176321fcf44af2b5a4103634b1908f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.624301 28520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b509f940a02545019c526e45e8372c97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.624939 28515 tablet_service.cc:1511] Processing CreateTablet for tablet 2d1073b726ef4cbeac6c874fc8863e90 (DEFAULT_TABLE table=default.Table_bbf208e6e39a47349ab50198f178974e [id=8e4cf344c21640728c419a084f14c7b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.625496 28514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e25027f423144b088dad955c7282921. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.625765 28510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fa1a27fc9304e18b96cad9f923779f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.625994 28515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d1073b726ef4cbeac6c874fc8863e90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.626238 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa93f065b4d64e7b85e30f0690fae541. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.626595 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f83bb8c3c504d5f9affbaa8be3f1508. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.626863 28512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2d54a901e9f4d5c89700979ce0618bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.627722 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 340f6bbf18eb412ca3afec237ff7752d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.628026 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80e99a85e83c46afa367e59e5ddb7dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.629288 28518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe1139c409e84070a2f496a85545e8d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.630606 28517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054a7afb3b644aa49df78302cf8f02c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.630513 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 438663f365e34fab87e34f434920cecf (DEFAULT_TABLE table=default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.631207 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 438663f365e34fab87e34f434920cecf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.634615 28519 tablet_service.cc:1511] Processing CreateTablet for tablet e5f7eaa7d218440ca2c35340c5c87814 (DEFAULT_TABLE table=default.Table_9a78338c3e414b0abea1281082202f57 [id=5697846ccd9c4c659755a0f9b8a66b72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.636068 28519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5f7eaa7d218440ca2c35340c5c87814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.637719 28527 tablet_service.cc:1511] Processing CreateTablet for tablet ba53ad1988064049a61fcca3a85f8779 (DEFAULT_TABLE table=default.Table_2a3bcaddd3034d67a5ea7c78f0ea9e7c [id=6c30b030a60b47e2be48dfe6e9cfdfdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.641719 28659 tablet_service.cc:1511] Processing CreateTablet for tablet fc295a8e81b64859915a6335f8fb8d8e (DEFAULT_TABLE table=default.Table_21633c55ca8c4614a2a0b241a0b89ee1 [id=411f4d0d27f94aa9842287ba693b2a01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.642107 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b968a8d4014341ca9c5964263721ac79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.642442 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc295a8e81b64859915a6335f8fb8d8e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:26.642390 28618 leader_election.cc:343] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Not found: Tablet not found: 668c6c6f2034484ca5ccdd51c22dea1d
I20260618 13:59:26.642563 28618 leader_election.cc:304] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:26.642858 29186 raft_consensus.cc:493] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:26.642998 29186 raft_consensus.cc:515] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.643714 29186 leader_election.cc:290] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:26.643657 29041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 451 } } ignore_live_leader: false dest_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" is_pre_election: true
I20260618 13:59:26.643862 29186 raft_consensus.cc:2749] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:26.644047 28647 tablet_service.cc:1511] Processing CreateTablet for tablet 85b2f34bc17b4a27a91d6cafa9a96c48 (DEFAULT_TABLE table=default.Table_19cfaeff1d174399b77939349481eb5a [id=0cfca3de151f4100b6003fa7b76f0e97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.644418 28647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85b2f34bc17b4a27a91d6cafa9a96c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.648682 28225 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_176a739c87ee45798e2470f8ec00d632" } modify_external_catalogs: true
I20260618 13:59:26.648873 28225 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_176a739c87ee45798e2470f8ec00d632" } modify_external_catalogs: true
I20260618 13:59:26.652252 28222 master_service.cc:960] ReplaceTablet: received request to replace tablet 41107eb1d9164a9c98060003b4027412 from {username='slave'} at 127.0.0.1:45138
W20260618 13:59:26.652559 28617 leader_election.cc:343] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 95db5ddaf5d5486988750efed5972eb6
I20260618 13:59:26.652993 28227 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_8c1fdd54393c4ad383e72a07837ead2a" } modify_external_catalogs: true
I20260618 13:59:26.653185 28227 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_8c1fdd54393c4ad383e72a07837ead2a" } modify_external_catalogs: true
I20260618 13:59:26.654647 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_9763de0aea1d44d785311a303abb1d27"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:26.655799 28220 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_c2afd6643b7b49b7992384135658ddfc" } modify_external_catalogs: true
I20260618 13:59:26.663144 28220 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_c2afd6643b7b49b7992384135658ddfc" } modify_external_catalogs: true
W20260618 13:59:26.663580 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9763de0aea1d44d785311a303abb1d27 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:26.665261 28511 tablet_service.cc:1511] Processing CreateTablet for tablet 55ae29c3b7ed44cfa659130745037aa6 (DEFAULT_TABLE table=default.Table_5d485acde41448d984aa79fa58b1866b [id=726324d365b94ff6bac24030222ca34f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.665556 28522 tablet_service.cc:1511] Processing CreateTablet for tablet fc1968e5ce2e42a5b836c90b0904f158 (DEFAULT_TABLE table=default.Table_dfe94c1f858644c98dba7db12d019477 [id=ea67f187738d41c993153f788e016e73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.665843 28513 tablet_service.cc:1558] Processing DeleteTablet for tablet 411b0c333ae7450f9ada90afee553615 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:26.666077 28801 tablet_replica.cc:333] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:26.666352 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet 591465e2b57247a2b224cc8b09225811 with delete_type TABLET_DATA_DELETED (replaced by tablet d4667fe48f0f4eff9c6c8baabdab58d4) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:26.666472 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba53ad1988064049a61fcca3a85f8779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.666623 28074 leader_election.cc:304] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e4d4f4566be42dfbafb7c0e40fb17ff; no voters: 7637285570574a6dbcef1de6f84ecac0, a950ab7cd66b4cb0b29592aa692c2dd0
I20260618 13:59:26.667344 28511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55ae29c3b7ed44cfa659130745037aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.667652 28529 tablet_service.cc:1511] Processing CreateTablet for tablet 97a9f949999446f09359629fe483bf67 (DEFAULT_TABLE table=default.Table_9424eec16a284672a0fe7ab3d0852b40 [id=070ea48af78346aa8ff43f2bb369730c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.669580 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 5df99b28e9ae4b6bbad0fe0dafd125d6 (table default.Table_2bf9616ba425458d8c7a4e9a6854d122 [id=e3f4e13202da42a786d3376ba662b31a]) successfully deleted
I20260618 13:59:26.670029 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet 02b5cc08ca0c4e719e0414b345211ba5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:26.670274 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet da4d4f03911f4ac099ecc2fa1d8af494 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da4d4f03911f4ac099ecc2fa1d8af494 already in progress: creating tablet
I20260618 13:59:26.670498 28516 tablet_service.cc:1511] Processing CreateTablet for tablet 7766014ee9814f01ae7ac3e849ca3d50 (DEFAULT_TABLE table=default.Table_c528008fd94b4fc8b7341f5b23dff76b [id=516d3679e0e84ab28429e59b6895db4e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:26.670578 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 827cc0987bd541759e334fb620f04ed5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 827cc0987bd541759e334fb620f04ed5 already in progress: creating tablet
I20260618 13:59:26.671149 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97a9f949999446f09359629fe483bf67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.671725 28513 tablet_service.cc:1511] Processing CreateTablet for tablet c38a2156a55e4e5eb91d101e9a61eecd (DEFAULT_TABLE table=default.Table_dc44ae6b83d844b6a16ce10ba6dce0ad [id=2e7a199c805d4453b2137b5fa69d0b4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.672031 28513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c38a2156a55e4e5eb91d101e9a61eecd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.673650 28229 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_bfcd14e3d49e4de09270ed85ec9cde4c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:26.674214 28229 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bfcd14e3d49e4de09270ed85ec9cde4c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:26.674999 28797 tablet_bootstrap.cc:492] T bab50e1f17a74eee8cad942525002996 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:26.676821 29186 raft_consensus.cc:493] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:26.677243 28797 ts_tablet_manager.cc:1403] T bab50e1f17a74eee8cad942525002996 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.295s user 0.043s sys 0.095s
I20260618 13:59:26.678045 28801 raft_consensus.cc:2243] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:26.678406 28663 tablet_service.cc:1511] Processing CreateTablet for tablet f1663f9d92e54f1caf3c7137412e495c (DEFAULT_TABLE table=default.Table_0a074f8e7c5443959918f5effc28e3d7 [id=e78c685d531d4d1b831e33f2b059b90c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.678643 28388 tablet_service.cc:1511] Processing CreateTablet for tablet 85b2f34bc17b4a27a91d6cafa9a96c48 (DEFAULT_TABLE table=default.Table_19cfaeff1d174399b77939349481eb5a [id=0cfca3de151f4100b6003fa7b76f0e97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.678825 29186 raft_consensus.cc:515] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.679044 28652 tablet_service.cc:1511] Processing CreateTablet for tablet d15b1b603f024f3295eb07c98991076d (DEFAULT_TABLE table=default.Table_7fa3da42543d4978aac5ac383ed10056 [id=9102540068234d1993fa164035dabc6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.679405 28656 tablet_service.cc:1511] Processing CreateTablet for tablet eebebb79db644535b54a48e4dee89d59 (DEFAULT_TABLE table=default.Table_84c59c9ab3f54fb4930771088e194560 [id=ee8aa366b2d1461099cb328367424ebd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.680265 28516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7766014ee9814f01ae7ac3e849ca3d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.681428 28655 tablet_service.cc:1511] Processing CreateTablet for tablet f45556a59c424ca2bc30956566374c89 (DEFAULT_TABLE table=default.Table_9c526031302044508f50fead286b4465 [id=408b312e0fad42ae8fb0ab84abd85c0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.682472 28391 tablet_service.cc:1511] Processing CreateTablet for tablet fc295a8e81b64859915a6335f8fb8d8e (DEFAULT_TABLE table=default.Table_21633c55ca8c4614a2a0b241a0b89ee1 [id=411f4d0d27f94aa9842287ba693b2a01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.682626 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d5e75b4a8247b29d69d07fad5ca5db" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:26.683228 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_0fbace1c1aa746a6a2a57c878631218b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:26.683610 28652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d15b1b603f024f3295eb07c98991076d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:26.683784 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0fbace1c1aa746a6a2a57c878631218b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:26.684073 28617 leader_election.cc:343] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: b0c9d4131ea24809ac081e7a73a21e1c
I20260618 13:59:26.684541 28221 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_0a13b90182654346acbbc8f47a56436d" } modify_external_catalogs: true
I20260618 13:59:26.684717 28221 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_0a13b90182654346acbbc8f47a56436d" } modify_external_catalogs: true
I20260618 13:59:26.686689 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc1968e5ce2e42a5b836c90b0904f158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.686771 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a0691de253b4b008e5d995fd6b876c2" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:26.687199 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eebebb79db644535b54a48e4dee89d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.688056 28521 tablet_service.cc:1511] Processing CreateTablet for tablet b968a8d4014341ca9c5964263721ac79 (DEFAULT_TABLE table=default.Table_b8741a7fd29f4e78a948b1c419595a85 [id=6f4d908593ae40f09e2c94cef218e1fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.688199 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1663f9d92e54f1caf3c7137412e495c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.688401 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f45556a59c424ca2bc30956566374c89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.689407 28228 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_90dfb0d3edbf419bb198321df1f067dc" } new_table_name: "default.Table_5159a242b12044fc9a0b3e535ea40050" modify_external_catalogs: true
I20260618 13:59:26.691322 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_83d021ed148243ffb453cb5e2484b7b4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:26.691370 28654 tablet_service.cc:1511] Processing CreateTablet for tablet c01c8463055e4526b3c75190066f30b6 (DEFAULT_TABLE table=default.Table_7e5d58ad3e8043919b66e8890819535c [id=96aff34b68c1494f9228a11eb38de4c7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:26.691825 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_83d021ed148243ffb453cb5e2484b7b4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:26.693799 29186 leader_election.cc:290] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:26.696399 28801 raft_consensus.cc:2272] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:26.697068 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c01c8463055e4526b3c75190066f30b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.696979 28797 raft_consensus.cc:359] T bab50e1f17a74eee8cad942525002996 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.697160 28797 raft_consensus.cc:385] T bab50e1f17a74eee8cad942525002996 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:26.697392 28797 raft_consensus.cc:740] T bab50e1f17a74eee8cad942525002996 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:26.697646 28797 consensus_queue.cc:260] T bab50e1f17a74eee8cad942525002996 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.698202 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85b2f34bc17b4a27a91d6cafa9a96c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.698662 28797 ts_tablet_manager.cc:1434] T bab50e1f17a74eee8cad942525002996 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.021s user 0.001s sys 0.001s
W20260618 13:59:26.701287 28346 leader_election.cc:343] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:26.701476 28346 leader_election.cc:304] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:26.701884 28346 leader_election.cc:343] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:26.702019 28346 leader_election.cc:304] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:26.703385 28384 tablet_service.cc:1511] Processing CreateTablet for tablet eebebb79db644535b54a48e4dee89d59 (DEFAULT_TABLE table=default.Table_84c59c9ab3f54fb4930771088e194560 [id=ee8aa366b2d1461099cb328367424ebd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.703786 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eebebb79db644535b54a48e4dee89d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.704010 28373 tablet_service.cc:1511] Processing CreateTablet for tablet f1663f9d92e54f1caf3c7137412e495c (DEFAULT_TABLE table=default.Table_0a074f8e7c5443959918f5effc28e3d7 [id=e78c685d531d4d1b831e33f2b059b90c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.704809 28376 tablet_service.cc:1511] Processing CreateTablet for tablet d15b1b603f024f3295eb07c98991076d (DEFAULT_TABLE table=default.Table_7fa3da42543d4978aac5ac383ed10056 [id=9102540068234d1993fa164035dabc6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.705135 28376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d15b1b603f024f3295eb07c98991076d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.705595 28373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1663f9d92e54f1caf3c7137412e495c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.706195 28374 tablet_service.cc:1511] Processing CreateTablet for tablet f45556a59c424ca2bc30956566374c89 (DEFAULT_TABLE table=default.Table_9c526031302044508f50fead286b4465 [id=408b312e0fad42ae8fb0ab84abd85c0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.706526 28374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f45556a59c424ca2bc30956566374c89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.706629 28797 tablet_bootstrap.cc:492] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:26.707412 28934 raft_consensus.cc:2749] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:26.707604 28934 raft_consensus.cc:2749] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:26.718258 28521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b968a8d4014341ca9c5964263721ac79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.721416 28372 tablet_service.cc:1511] Processing CreateTablet for tablet 8af2bec820b842b4bf6b57482007bd6a (DEFAULT_TABLE table=default.Table_9a10a9aa97574e7c89586d1e4e863171 [id=af1859c644b44c1d831314d27a494f94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.721848 28372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8af2bec820b842b4bf6b57482007bd6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.722970 28381 tablet_service.cc:1511] Processing CreateTablet for tablet f2b21dcf74fa4264947ca43cc0026e76 (DEFAULT_TABLE table=default.Table_c35470a550534fa2ad072f932e981df5 [id=2797c7fefad74c1e82ab31fb9ad4db70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.723039 28653 tablet_service.cc:1511] Processing CreateTablet for tablet 8af2bec820b842b4bf6b57482007bd6a (DEFAULT_TABLE table=default.Table_9a10a9aa97574e7c89586d1e4e863171 [id=af1859c644b44c1d831314d27a494f94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.723305 28381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2b21dcf74fa4264947ca43cc0026e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.723528 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8af2bec820b842b4bf6b57482007bd6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.723734 28657 tablet_service.cc:1511] Processing CreateTablet for tablet f2b21dcf74fa4264947ca43cc0026e76 (DEFAULT_TABLE table=default.Table_c35470a550534fa2ad072f932e981df5 [id=2797c7fefad74c1e82ab31fb9ad4db70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.724072 28377 tablet_service.cc:1511] Processing CreateTablet for tablet 5b9f9649587b4c31951204b8bebe7ed8 (DEFAULT_TABLE table=default.Table_f7b8e1561d67469b9a3709475d661de9 [id=caa2a3299e864159b261740eecc0f64d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.724157 28659 tablet_service.cc:1511] Processing CreateTablet for tablet 5b9f9649587b4c31951204b8bebe7ed8 (DEFAULT_TABLE table=default.Table_f7b8e1561d67469b9a3709475d661de9 [id=caa2a3299e864159b261740eecc0f64d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.724385 28377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b9f9649587b4c31951204b8bebe7ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.725418 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac58e939730c4af681c2f08bcfbb6602" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:26.726063 28387 tablet_service.cc:1511] Processing CreateTablet for tablet ed648dcf882340388155068d7a5a7869 (DEFAULT_TABLE table=default.Table_8c9c9a6eea174bd68d8fb9d9ebaaf603 [id=185e7792e79644a2aced2b949fca635b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.726905 28387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed648dcf882340388155068d7a5a7869. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.727838 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2b21dcf74fa4264947ca43cc0026e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.727010 28662 tablet_service.cc:1511] Processing CreateTablet for tablet 8f7b66652f464893ad812d27e883a9a4 (DEFAULT_TABLE table=default.Table_697444611a2643bfb1fe30fc545c47f3 [id=4aece4c39f65473494e823b598e81bd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.728336 28385 tablet_service.cc:1511] Processing CreateTablet for tablet c01c8463055e4526b3c75190066f30b6 (DEFAULT_TABLE table=default.Table_7e5d58ad3e8043919b66e8890819535c [id=96aff34b68c1494f9228a11eb38de4c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.728647 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c01c8463055e4526b3c75190066f30b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.731873 28801 ts_tablet_manager.cc:1916] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:26.732429 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f7b66652f464893ad812d27e883a9a4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:26.734687 28228 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6b8f273def3e4b7caa4d624cf87a106c consensus state has no leader
I20260618 13:59:26.742187 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc295a8e81b64859915a6335f8fb8d8e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:26.751152 28483 leader_election.cc:343] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 8be4374ae7cf4907940ae90299bf9ba0
I20260618 13:59:26.758520 28652 tablet_service.cc:1511] Processing CreateTablet for tablet ed648dcf882340388155068d7a5a7869 (DEFAULT_TABLE table=default.Table_8c9c9a6eea174bd68d8fb9d9ebaaf603 [id=185e7792e79644a2aced2b949fca635b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.760023 28797 tablet_bootstrap.cc:654] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:26.762761 28226 master_service.cc:960] ReplaceTablet: received request to replace tablet 75824a5d2f7e47838b60e6a2b785c6b2 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:26.766844 28506 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:34078 (request call id 100) took 1068 ms. Trace:
I20260618 13:59:26.767292 28649 tablet_service.cc:1511] Processing CreateTablet for tablet 9559c5f2c9e94e858093dd2f0865c865 (DEFAULT_TABLE table=default.Table_38a3b325a9514a3d829c6ba288f9673a [id=8f0879e0c0b74edfa8ae99d11cda07a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.770648 28506 rpcz_store.cc:276] 0618 13:59:25.636765 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.636825 (+ 60us) service_pool.cc:224] Handling call
0618 13:59:26.705443 (+1068618us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260618 13:59:26.772621 28483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 60120.1us
I20260618 13:59:26.774631 28663 tablet_service.cc:1511] Processing CreateTablet for tablet d4667fe48f0f4eff9c6c8baabdab58d4 (DEFAULT_TABLE table=default.Table_0a85d79af65f48a3803bfd1248593a11 [id=2f4d62be772a421ab777fd23be58b719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.780359 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b9f9649587b4c31951204b8bebe7ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.786006 28649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9559c5f2c9e94e858093dd2f0865c865. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:26.787204 28346 leader_election.cc:343] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:26.788224 28346 leader_election.cc:304] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:26.788751 29183 raft_consensus.cc:2749] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:26.794085 28374 tablet_service.cc:1511] Processing CreateTablet for tablet 9559c5f2c9e94e858093dd2f0865c865 (DEFAULT_TABLE table=default.Table_38a3b325a9514a3d829c6ba288f9673a [id=8f0879e0c0b74edfa8ae99d11cda07a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.797075 28384 tablet_service.cc:1511] Processing CreateTablet for tablet d4667fe48f0f4eff9c6c8baabdab58d4 (DEFAULT_TABLE table=default.Table_0a85d79af65f48a3803bfd1248593a11 [id=2f4d62be772a421ab777fd23be58b719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.797458 28381 tablet_service.cc:1511] Processing CreateTablet for tablet 8f7b66652f464893ad812d27e883a9a4 (DEFAULT_TABLE table=default.Table_697444611a2643bfb1fe30fc545c47f3 [id=4aece4c39f65473494e823b598e81bd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.797976 28374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9559c5f2c9e94e858093dd2f0865c865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.798784 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 3a783590159f4a80964fbbcb4a24eef9 (DEFAULT_TABLE table=default.Table_abe180e115a04052ae89cc92d3c1c1e6 [id=4cbcd521f63047a38f5a9b9def56df7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.801486 28652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed648dcf882340388155068d7a5a7869. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:26.801479 28618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 193303us
W20260618 13:59:26.801796 28618 leader_election.cc:343] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:26.802044 28618 leader_election.cc:304] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:26.802578 28381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f7b66652f464893ad812d27e883a9a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.803016 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be4374ae7cf4907940ae90299bf9ba0" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:26.803104 29208 raft_consensus.cc:2749] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:26.806668 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4667fe48f0f4eff9c6c8baabdab58d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.809854 28800 ts_tablet_manager.cc:1929] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:26.810025 28800 log.cc:1199] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/5df99b28e9ae4b6bbad0fe0dafd125d6
I20260618 13:59:26.810384 28800 ts_tablet_manager.cc:1950] T 5df99b28e9ae4b6bbad0fe0dafd125d6 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:26.811414 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4667fe48f0f4eff9c6c8baabdab58d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.814025 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 5df99b28e9ae4b6bbad0fe0dafd125d6 (table default.Table_2bf9616ba425458d8c7a4e9a6854d122 [id=e3f4e13202da42a786d3376ba662b31a]) successfully deleted
I20260618 13:59:26.814599 28800 tablet_replica.cc:333] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:26.814869 28800 raft_consensus.cc:2243] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:26.815097 28800 raft_consensus.cc:2272] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:26.815712 28800 ts_tablet_manager.cc:1916] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:26.825212 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b88ed7cf79c4929a8704f66691cbed4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:26.825625 28799 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.581s user 0.023s sys 0.034s
W20260618 13:59:26.826022 28799 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.582s user 0.023s sys 0.035s
I20260618 13:59:26.827039 28799 tablet_bootstrap.cc:492] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:26.827335 28799 ts_tablet_manager.cc:1403] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.706s user 0.025s sys 0.037s
I20260618 13:59:26.828699 28799 raft_consensus.cc:359] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:26.829922 28799 raft_consensus.cc:385] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:26.830127 28799 raft_consensus.cc:740] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:26.829792 28802 ts_tablet_manager.cc:1929] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:26.830587 28802 log.cc:1199] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/4c2b924a096b479cad31c29bf7c99172
I20260618 13:59:26.831092 28802 ts_tablet_manager.cc:1950] T 4c2b924a096b479cad31c29bf7c99172 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:26.832176 28802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59166 (request call id 343) took 1656 ms. Trace:
I20260618 13:59:26.832373 28802 rpcz_store.cc:276] 0618 13:59:25.175329 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.175603 (+ 274us) service_pool.cc:224] Handling call
0618 13:59:26.563972 (+1388369us) ts_tablet_manager.cc:1151] Deleting tablet 4c2b924a096b479cad31c29bf7c99172
0618 13:59:26.563982 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:26.819926 (+255944us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:26.829706 (+ 9780us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:26.832105 (+ 2399us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1388011}
I20260618 13:59:26.831250 28799 consensus_queue.cc:260] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:26.833966 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 4c2b924a096b479cad31c29bf7c99172 (table default.Table_3deacb2c9668491a85bbb09ae3f57d66 [id=431a660bfbed46248a603dc7781cf543]) successfully deleted
I20260618 13:59:26.834760 28799 ts_tablet_manager.cc:1434] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260618 13:59:26.835090 28799 tablet_bootstrap.cc:492] T 37a6764a5e024903848561c9935b1a43 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:26.837183 28802 tablet_replica.cc:333] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:26.838058 29190 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260618 13:59:26.815163 28484 leader_election.cc:343] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:26.842730 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a783590159f4a80964fbbcb4a24eef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.844005 28802 raft_consensus.cc:2243] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:26.844197 28802 raft_consensus.cc:2272] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:26.844738 28802 ts_tablet_manager.cc:1916] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:26.846486 28484 leader_election.cc:304] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:26.847220 28799 tablet_bootstrap.cc:654] T 37a6764a5e024903848561c9935b1a43 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:26.847879 29185 raft_consensus.cc:2749] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:26.851428 28797 tablet_bootstrap.cc:492] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:26.851697 28797 ts_tablet_manager.cc:1403] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.148s user 0.004s sys 0.009s
I20260618 13:59:26.852577 28802 ts_tablet_manager.cc:1929] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:26.852690 28802 log.cc:1199] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/516421cd843b44769efa2b555697c065
I20260618 13:59:26.852785 28797 raft_consensus.cc:359] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.853039 28802 ts_tablet_manager.cc:1950] T 516421cd843b44769efa2b555697c065 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:26.853113 28797 raft_consensus.cc:385] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:26.853303 28797 raft_consensus.cc:740] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:26.853655 28510 tablet_service.cc:1558] Processing DeleteTablet for tablet 591465e2b57247a2b224cc8b09225811 with delete_type TABLET_DATA_DELETED (replaced by tablet d4667fe48f0f4eff9c6c8baabdab58d4) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:26.853605 28797 consensus_queue.cc:260] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.853899 28802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59166 (request call id 345) took 1678 ms. Trace:
I20260618 13:59:26.854004 28802 rpcz_store.cc:276] 0618 13:59:25.175577 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.176139 (+ 562us) service_pool.cc:224] Handling call
0618 13:59:26.837098 (+1660959us) ts_tablet_manager.cc:1151] Deleting tablet 516421cd843b44769efa2b555697c065
0618 13:59:26.837108 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:26.849300 (+ 12192us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:26.852505 (+ 3205us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:26.853866 (+ 1361us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1660768}
I20260618 13:59:26.854177 28797 ts_tablet_manager.cc:1434] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:26.855017 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 516421cd843b44769efa2b555697c065 (table default.Table_c5c65f038c8f4f2783a705ffcc320629 [id=b06c685b8107494392219d12540a3a6e]) successfully deleted
I20260618 13:59:26.858309 28799 tablet_bootstrap.cc:492] T 37a6764a5e024903848561c9935b1a43 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:26.858475 28799 ts_tablet_manager.cc:1403] T 37a6764a5e024903848561c9935b1a43 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.004s
I20260618 13:59:26.858651 28516 tablet_service.cc:1511] Processing CreateTablet for tablet 85b2f34bc17b4a27a91d6cafa9a96c48 (DEFAULT_TABLE table=default.Table_19cfaeff1d174399b77939349481eb5a [id=0cfca3de151f4100b6003fa7b76f0e97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.858893 28797 tablet_bootstrap.cc:492] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:26.859206 28799 raft_consensus.cc:359] T 37a6764a5e024903848561c9935b1a43 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:26.859402 28799 raft_consensus.cc:385] T 37a6764a5e024903848561c9935b1a43 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:26.859481 28799 raft_consensus.cc:740] T 37a6764a5e024903848561c9935b1a43 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:26.859722 28799 consensus_queue.cc:260] T 37a6764a5e024903848561c9935b1a43 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:26.861235 28618 leader_election.cc:343] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Not found: Tablet not found: 95db5ddaf5d5486988750efed5972eb6
I20260618 13:59:26.861382 28618 leader_election.cc:304] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:26.861934 29208 raft_consensus.cc:2749] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:26.864869 28799 ts_tablet_manager.cc:1434] T 37a6764a5e024903848561c9935b1a43 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
I20260618 13:59:26.864888 28797 tablet_bootstrap.cc:654] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:26.867959 28524 tablet_service.cc:1511] Processing CreateTablet for tablet d15b1b603f024f3295eb07c98991076d (DEFAULT_TABLE table=default.Table_7fa3da42543d4978aac5ac383ed10056 [id=9102540068234d1993fa164035dabc6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.868422 28799 tablet_bootstrap.cc:492] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:26.870723 28618 leader_election.cc:343] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Not found: Tablet not found: b0c9d4131ea24809ac081e7a73a21e1c
I20260618 13:59:26.870904 28618 leader_election.cc:304] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:26.871233 28799 tablet_bootstrap.cc:654] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:26.871421 29208 raft_consensus.cc:2749] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:26.872298 28802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59166 (request call id 347) took 1617 ms. Trace:
I20260618 13:59:26.872417 28802 rpcz_store.cc:276] 0618 13:59:25.254812 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.254870 (+ 58us) service_pool.cc:224] Handling call
0618 13:59:26.872151 (+1617281us) ts_tablet_manager.cc:1151] Deleting tablet aa8cf927bed9437086efd8e0f21ba26a
0618 13:59:26.872162 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:26.872262 (+ 100us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1617017}
I20260618 13:59:26.872879 28800 ts_tablet_manager.cc:1929] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:26.872997 28802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59166 (request call id 348) took 1595 ms. Trace:
I20260618 13:59:26.873016 28800 log.cc:1199] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/b1f3dc9c3d874ca4ae42fd5e6d84fcb0
W20260618 13:59:26.872997 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet aa8cf927bed9437086efd8e0f21ba26a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa8cf927bed9437086efd8e0f21ba26a already in progress: creating tablet
I20260618 13:59:26.873113 28802 rpcz_store.cc:276] 0618 13:59:25.277751 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.277813 (+ 62us) service_pool.cc:224] Handling call
0618 13:59:26.872863 (+1595050us) ts_tablet_manager.cc:1151] Deleting tablet be59f988f0e34b9480f859cf88b9c223
0618 13:59:26.872872 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:26.872960 (+ 88us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1591361}
I20260618 13:59:26.873386 28800 ts_tablet_manager.cc:1950] T b1f3dc9c3d874ca4ae42fd5e6d84fcb0 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
W20260618 13:59:26.873508 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet be59f988f0e34b9480f859cf88b9c223 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet be59f988f0e34b9480f859cf88b9c223 already in progress: creating tablet
I20260618 13:59:26.873557 28802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59166 (request call id 346) took 1620 ms. Trace:
I20260618 13:59:26.873658 28802 rpcz_store.cc:276] 0618 13:59:25.252767 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.273702 (+ 20935us) service_pool.cc:224] Handling call
0618 13:59:26.873440 (+1599738us) ts_tablet_manager.cc:1151] Deleting tablet 316f470ea82f42fca48e4e96d49c9574
0618 13:59:26.873448 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:26.873527 (+ 79us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1591686}
I20260618 13:59:26.874097 28802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59166 (request call id 352) took 1495 ms. Trace:
I20260618 13:59:26.874207 28802 rpcz_store.cc:276] 0618 13:59:25.378160 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.378207 (+ 47us) service_pool.cc:224] Handling call
0618 13:59:26.873989 (+1495782us) ts_tablet_manager.cc:1151] Deleting tablet dd2385e6752d49429422fc854539e9c6
0618 13:59:26.873997 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:26.874070 (+ 73us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1492215}
I20260618 13:59:26.874596 28800 tablet_replica.cc:333] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:26.874681 28802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59166 (request call id 355) took 1444 ms. Trace:
I20260618 13:59:26.874779 28802 rpcz_store.cc:276] 0618 13:59:25.430252 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.430307 (+ 55us) service_pool.cc:224] Handling call
0618 13:59:26.874558 (+1444251us) ts_tablet_manager.cc:1151] Deleting tablet 18d8b609b96a4e81b57f69bff522eeea
0618 13:59:26.874567 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:26.874652 (+ 85us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1443970}
I20260618 13:59:26.874894 28800 raft_consensus.cc:2243] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:26.875125 28800 raft_consensus.cc:2272] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:26.875229 28802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59166 (request call id 356) took 1435 ms. Trace:
I20260618 13:59:26.875339 28802 rpcz_store.cc:276] 0618 13:59:25.439337 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.439388 (+ 51us) service_pool.cc:224] Handling call
0618 13:59:26.875114 (+1435726us) ts_tablet_manager.cc:1151] Deleting tablet fe750fc5d998448caeab0a6c808dfba7
0618 13:59:26.875122 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:26.875199 (+ 77us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1435506}
W20260618 13:59:26.875633 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 316f470ea82f42fca48e4e96d49c9574 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 316f470ea82f42fca48e4e96d49c9574 already in progress: creating tablet
I20260618 13:59:26.875782 28802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59166 (request call id 359) took 1434 ms. Trace:
I20260618 13:59:26.875802 28800 ts_tablet_manager.cc:1916] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:26.875890 28802 rpcz_store.cc:276] 0618 13:59:25.441562 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.442243 (+ 681us) service_pool.cc:224] Handling call
0618 13:59:26.875665 (+1433422us) ts_tablet_manager.cc:1151] Deleting tablet 7d4144b1b5ea4e968672616bc22bead8
0618 13:59:26.875674 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:26.875752 (+ 78us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1433184}
W20260618 13:59:26.875995 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet dd2385e6752d49429422fc854539e9c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd2385e6752d49429422fc854539e9c6 already in progress: creating tablet
W20260618 13:59:26.876390 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 18d8b609b96a4e81b57f69bff522eeea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 18d8b609b96a4e81b57f69bff522eeea already in progress: creating tablet
I20260618 13:59:26.876694 28802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59166 (request call id 364) took 1368 ms. Trace:
W20260618 13:59:26.876765 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet fe750fc5d998448caeab0a6c808dfba7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe750fc5d998448caeab0a6c808dfba7 already in progress: creating tablet
I20260618 13:59:26.876875 28802 rpcz_store.cc:276] 0618 13:59:25.507747 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.508067 (+ 320us) service_pool.cc:224] Handling call
0618 13:59:26.876509 (+1368442us) ts_tablet_manager.cc:1151] Deleting tablet 0463fa2e30554abcbfc8f37b702e81d4
0618 13:59:26.876517 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:26.876601 (+ 84us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1368227}
W20260618 13:59:26.877646 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 7d4144b1b5ea4e968672616bc22bead8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d4144b1b5ea4e968672616bc22bead8 already in progress: creating tablet
I20260618 13:59:26.877756 28802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59166 (request call id 363) took 1370 ms. Trace:
I20260618 13:59:26.877861 28802 rpcz_store.cc:276] 0618 13:59:25.507610 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.514580 (+ 6970us) service_pool.cc:224] Handling call
0618 13:59:26.877628 (+1363048us) ts_tablet_manager.cc:1151] Deleting tablet b6dd422c05324c989e29b20ae2afabad
0618 13:59:26.877639 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:26.877726 (+ 87us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1362447}
W20260618 13:59:26.878153 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 0463fa2e30554abcbfc8f37b702e81d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0463fa2e30554abcbfc8f37b702e81d4 already in progress: creating tablet
I20260618 13:59:26.878504 28516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85b2f34bc17b4a27a91d6cafa9a96c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.879228 28797 tablet_bootstrap.cc:492] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:26.879381 28797 ts_tablet_manager.cc:1403] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.002s
I20260618 13:59:26.880147 28797 raft_consensus.cc:359] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.880330 28797 raft_consensus.cc:385] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:26.880308 29183 raft_consensus.cc:493] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:26.880411 28797 raft_consensus.cc:740] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:26.880458 29183 raft_consensus.cc:515] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:26.880626 28797 consensus_queue.cc:260] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.881094 29183 leader_election.cc:290] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:26.881448 28797 ts_tablet_manager.cc:1434] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:26.881836 28797 tablet_bootstrap.cc:492] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:26.882644 28525 tablet_service.cc:1511] Processing CreateTablet for tablet eebebb79db644535b54a48e4dee89d59 (DEFAULT_TABLE table=default.Table_84c59c9ab3f54fb4930771088e194560 [id=ee8aa366b2d1461099cb328367424ebd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.883324 28528 tablet_service.cc:1511] Processing CreateTablet for tablet c01c8463055e4526b3c75190066f30b6 (DEFAULT_TABLE table=default.Table_7e5d58ad3e8043919b66e8890819535c [id=96aff34b68c1494f9228a11eb38de4c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.883559 28797 tablet_bootstrap.cc:654] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:26.884162 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b9c0754beb4e0e8ed3a7c1e9f368a9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:26.887315 28802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59166 (request call id 373) took 1225 ms. Trace:
I20260618 13:59:26.887282 28510 tablet_service.cc:1511] Processing CreateTablet for tablet fc295a8e81b64859915a6335f8fb8d8e (DEFAULT_TABLE table=default.Table_21633c55ca8c4614a2a0b241a0b89ee1 [id=411f4d0d27f94aa9842287ba693b2a01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.887424 28802 rpcz_store.cc:276] 0618 13:59:25.661494 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.666345 (+ 4851us) service_pool.cc:224] Handling call
0618 13:59:26.887125 (+1220780us) ts_tablet_manager.cc:1151] Deleting tablet d667ea5f029f46d89fcb3be03069b97b
0618 13:59:26.887138 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:26.887281 (+ 143us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1213787}
W20260618 13:59:26.887996 28346 leader_election.cc:343] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:26.888120 29208 raft_consensus.cc:493] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:26.888283 29208 raft_consensus.cc:515] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.889159 29208 leader_election.cc:290] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:26.889314 28802 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59166 (request call id 372) took 1228 ms. Trace:
I20260618 13:59:26.889415 28802 rpcz_store.cc:276] 0618 13:59:25.661347 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:25.666411 (+ 5064us) service_pool.cc:224] Handling call
0618 13:59:26.889177 (+1222766us) ts_tablet_manager.cc:1151] Deleting tablet b0285c75c632457383e81805ab036aaf
0618 13:59:26.889188 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:26.889278 (+ 90us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1116361}
I20260618 13:59:26.889608 28523 tablet_service.cc:1511] Processing CreateTablet for tablet f45556a59c424ca2bc30956566374c89 (DEFAULT_TABLE table=default.Table_9c526031302044508f50fead286b4465 [id=408b312e0fad42ae8fb0ab84abd85c0b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:26.890512 28617 leader_election.cc:343] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 8be4374ae7cf4907940ae90299bf9ba0
I20260618 13:59:26.890873 28802 tablet_replica.cc:333] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:26.891103 28802 raft_consensus.cc:2243] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:26.891269 28802 raft_consensus.cc:2272] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:26.891836 28802 ts_tablet_manager.cc:1916] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:26.891959 28800 ts_tablet_manager.cc:1929] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:26.892091 28800 log.cc:1199] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/668c6c6f2034484ca5ccdd51c22dea1d
I20260618 13:59:26.892439 28800 ts_tablet_manager.cc:1950] T 668c6c6f2034484ca5ccdd51c22dea1d P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:26.893005 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet 8be4374ae7cf4907940ae90299bf9ba0 with delete_type TABLET_DATA_DELETED (replaced by tablet 092a9aba9c6e48f5be9cf5d55ba93678) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:26.893260 28515 tablet_service.cc:1511] Processing CreateTablet for tablet f1663f9d92e54f1caf3c7137412e495c (DEFAULT_TABLE table=default.Table_0a074f8e7c5443959918f5effc28e3d7 [id=e78c685d531d4d1b831e33f2b059b90c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.896344 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b9c0754beb4e0e8ed3a7c1e9f368a9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:26.897373 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be4374ae7cf4907940ae90299bf9ba0" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:26.897568 28548 raft_consensus.cc:2468] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:26.898275 28802 ts_tablet_manager.cc:1929] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:26.899418 28797 tablet_bootstrap.cc:492] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:26.899605 28797 ts_tablet_manager.cc:1403] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.002s
I20260618 13:59:26.900523 28797 raft_consensus.cc:359] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.900735 28797 raft_consensus.cc:385] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:26.901607 28797 raft_consensus.cc:740] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:26.902200 28797 consensus_queue.cc:260] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.904170 28800 tablet_replica.cc:333] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:26.904390 28800 raft_consensus.cc:2243] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:26.904579 28800 raft_consensus.cc:2272] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:26.898401 28802 log.cc:1199] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/ac58e939730c4af681c2f08bcfbb6602
I20260618 13:59:26.904994 28517 tablet_service.cc:1511] Processing CreateTablet for tablet 8af2bec820b842b4bf6b57482007bd6a (DEFAULT_TABLE table=default.Table_9a10a9aa97574e7c89586d1e4e863171 [id=af1859c644b44c1d831314d27a494f94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.905108 28800 ts_tablet_manager.cc:1916] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:26.905345 28802 ts_tablet_manager.cc:1950] T ac58e939730c4af681c2f08bcfbb6602 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:26.908003 28618 leader_election.cc:304] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:26.908174 28802 tablet_replica.cc:333] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:26.908419 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eebebb79db644535b54a48e4dee89d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.908402 28802 raft_consensus.cc:2243] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:26.908524 29208 raft_consensus.cc:2804] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:26.908603 28802 raft_consensus.cc:2272] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:26.908638 29208 raft_consensus.cc:493] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:26.909173 29208 raft_consensus.cc:3060] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:26.909965 28802 ts_tablet_manager.cc:1916] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:26.912101 28800 ts_tablet_manager.cc:1929] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:26.912016 29208 raft_consensus.cc:515] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.912221 28800 log.cc:1199] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/411b0c333ae7450f9ada90afee553615
I20260618 13:59:26.912670 28800 ts_tablet_manager.cc:1950] T 411b0c333ae7450f9ada90afee553615 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:26.913440 28514 tablet_service.cc:1511] Processing CreateTablet for tablet f2b21dcf74fa4264947ca43cc0026e76 (DEFAULT_TABLE table=default.Table_c35470a550534fa2ad072f932e981df5 [id=2797c7fefad74c1e82ab31fb9ad4db70]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:26.913568 28617 leader_election.cc:343] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 8be4374ae7cf4907940ae90299bf9ba0
I20260618 13:59:26.913776 28510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc295a8e81b64859915a6335f8fb8d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.914616 29208 leader_election.cc:290] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:26.915149 28797 ts_tablet_manager.cc:1434] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.015s user 0.003s sys 0.000s
I20260618 13:59:26.910624 28519 tablet_service.cc:1511] Processing CreateTablet for tablet ed648dcf882340388155068d7a5a7869 (DEFAULT_TABLE table=default.Table_8c9c9a6eea174bd68d8fb9d9ebaaf603 [id=185e7792e79644a2aced2b949fca635b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.921190 28800 tablet_replica.cc:333] T 3752f49c9fc64071a45e0ce4ff4cfef2 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
W20260618 13:59:26.921188 28346 leader_election.cc:343] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:26.921329 28346 leader_election.cc:304] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:26.921392 28802 ts_tablet_manager.cc:1929] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:26.921407 28800 raft_consensus.cc:2243] T 3752f49c9fc64071a45e0ce4ff4cfef2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:26.921491 28802 log.cc:1199] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/da4d4f03911f4ac099ecc2fa1d8af494
I20260618 13:59:26.921590 28800 raft_consensus.cc:2272] T 3752f49c9fc64071a45e0ce4ff4cfef2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:26.921701 29208 raft_consensus.cc:670] T 3752f49c9fc64071a45e0ce4ff4cfef2 P d6a169b16ced46ca9cfbc631709bb257: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopped
I20260618 13:59:26.921813 29183 raft_consensus.cc:2749] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:26.921860 28802 ts_tablet_manager.cc:1950] T da4d4f03911f4ac099ecc2fa1d8af494 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:26.922103 28797 tablet_bootstrap.cc:492] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:26.922437 28800 ts_tablet_manager.cc:1916] T 3752f49c9fc64071a45e0ce4ff4cfef2 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:26.922744 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be4374ae7cf4907940ae90299bf9ba0" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:26.922931 28548 raft_consensus.cc:3060] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:26.922986 28802 tablet_replica.cc:333] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:26.923200 28802 raft_consensus.cc:2243] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:26.923596 28802 raft_consensus.cc:2272] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:26.924145 28802 ts_tablet_manager.cc:1916] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:26.924631 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f45556a59c424ca2bc30956566374c89. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:26.925550 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet b6dd422c05324c989e29b20ae2afabad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6dd422c05324c989e29b20ae2afabad already in progress: creating tablet
W20260618 13:59:26.926035 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet d667ea5f029f46d89fcb3be03069b97b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d667ea5f029f46d89fcb3be03069b97b already in progress: creating tablet
I20260618 13:59:26.926106 28797 tablet_bootstrap.cc:654] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:26.926951 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet b0285c75c632457383e81805ab036aaf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0285c75c632457383e81805ab036aaf already in progress: creating tablet
I20260618 13:59:26.929926 28800 ts_tablet_manager.cc:1929] T 3752f49c9fc64071a45e0ce4ff4cfef2 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:26.930080 28800 log.cc:1199] T 3752f49c9fc64071a45e0ce4ff4cfef2 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/3752f49c9fc64071a45e0ce4ff4cfef2
I20260618 13:59:26.938566 28800 ts_tablet_manager.cc:1950] T 3752f49c9fc64071a45e0ce4ff4cfef2 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:26.938795 28512 tablet_service.cc:1511] Processing CreateTablet for tablet 5b9f9649587b4c31951204b8bebe7ed8 (DEFAULT_TABLE table=default.Table_f7b8e1561d67469b9a3709475d661de9 [id=caa2a3299e864159b261740eecc0f64d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.939810 28802 ts_tablet_manager.cc:1929] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:26.940205 28800 tablet_replica.cc:333] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:26.940436 28800 raft_consensus.cc:2243] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:26.940642 28800 raft_consensus.cc:2272] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:26.941265 28800 ts_tablet_manager.cc:1916] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:26.941681 28802 log.cc:1199] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/411b0c333ae7450f9ada90afee553615
I20260618 13:59:26.942318 28802 ts_tablet_manager.cc:1950] T 411b0c333ae7450f9ada90afee553615 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:26.942853 28515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1663f9d92e54f1caf3c7137412e495c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.945017 28797 tablet_bootstrap.cc:492] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:26.945159 28797 ts_tablet_manager.cc:1403] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.023s user 0.005s sys 0.006s
I20260618 13:59:26.946060 28797 raft_consensus.cc:359] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.946293 28797 raft_consensus.cc:385] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:26.946384 28797 raft_consensus.cc:740] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:26.948057 28797 consensus_queue.cc:260] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.949194 28519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed648dcf882340388155068d7a5a7869. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.952255 28548 raft_consensus.cc:2468] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:26.952862 28618 leader_election.cc:304] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:26.953364 29208 raft_consensus.cc:2804] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:26.953521 29208 raft_consensus.cc:697] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:26.953873 29208 consensus_queue.cc:237] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:26.958475 28800 ts_tablet_manager.cc:1929] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:26.958602 28800 log.cc:1199] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/dca54111fea047d9ae0fabe376d89382
I20260618 13:59:26.958986 28800 ts_tablet_manager.cc:1950] T dca54111fea047d9ae0fabe376d89382 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:26.960302 28800 tablet_replica.cc:333] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:26.960636 28800 raft_consensus.cc:2243] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:26.960847 28800 raft_consensus.cc:2272] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:26.961499 28800 ts_tablet_manager.cc:1916] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:26.974843 28800 ts_tablet_manager.cc:1929] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:26.975035 28800 log.cc:1199] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/95db5ddaf5d5486988750efed5972eb6
I20260618 13:59:26.975400 28800 ts_tablet_manager.cc:1950] T 95db5ddaf5d5486988750efed5972eb6 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:26.976645 28797 ts_tablet_manager.cc:1434] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260618 13:59:26.977103 28797 tablet_bootstrap.cc:492] T fca76774f119435899188313a33dad98 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:26.978881 28797 tablet_bootstrap.cc:654] T fca76774f119435899188313a33dad98 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:26.980232 28800 tablet_replica.cc:333] T 08a2dee3ac5a4bec96445a6001717eda P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:26.980438 28800 raft_consensus.cc:2243] T 08a2dee3ac5a4bec96445a6001717eda P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:26.980633 28800 raft_consensus.cc:2272] T 08a2dee3ac5a4bec96445a6001717eda P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:26.981194 28800 ts_tablet_manager.cc:1916] T 08a2dee3ac5a4bec96445a6001717eda P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:26.984647 28529 tablet_service.cc:1511] Processing CreateTablet for tablet 9559c5f2c9e94e858093dd2f0865c865 (DEFAULT_TABLE table=default.Table_38a3b325a9514a3d829c6ba288f9673a [id=8f0879e0c0b74edfa8ae99d11cda07a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.986147 28522 tablet_service.cc:1511] Processing CreateTablet for tablet d4667fe48f0f4eff9c6c8baabdab58d4 (DEFAULT_TABLE table=default.Table_0a85d79af65f48a3803bfd1248593a11 [id=2f4d62be772a421ab777fd23be58b719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:26.987708 28514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2b21dcf74fa4264947ca43cc0026e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.988484 28797 tablet_bootstrap.cc:492] T fca76774f119435899188313a33dad98 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:26.988649 28797 ts_tablet_manager.cc:1403] T fca76774f119435899188313a33dad98 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20260618 13:59:26.989452 28797 raft_consensus.cc:359] T fca76774f119435899188313a33dad98 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:26.989679 28797 raft_consensus.cc:385] T fca76774f119435899188313a33dad98 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:26.989770 28797 raft_consensus.cc:740] T fca76774f119435899188313a33dad98 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:26.989826 28800 ts_tablet_manager.cc:1929] T 08a2dee3ac5a4bec96445a6001717eda P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:26.990006 28800 log.cc:1199] T 08a2dee3ac5a4bec96445a6001717eda P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/08a2dee3ac5a4bec96445a6001717eda
I20260618 13:59:26.990036 28797 consensus_queue.cc:260] T fca76774f119435899188313a33dad98 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:26.990509 28800 ts_tablet_manager.cc:1950] T 08a2dee3ac5a4bec96445a6001717eda P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:26.990567 28797 ts_tablet_manager.cc:1434] T fca76774f119435899188313a33dad98 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:26.991307 28797 tablet_bootstrap.cc:492] T 0a51e830e3014726b1d1c2608973f036 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:26.991505 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d15b1b603f024f3295eb07c98991076d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:26.992811 28797 tablet_bootstrap.cc:654] T 0a51e830e3014726b1d1c2608973f036 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:26.995522 28800 tablet_replica.cc:333] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:26.995869 28800 raft_consensus.cc:2243] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:26.996812 28800 raft_consensus.cc:2272] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:26.999308 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4667fe48f0f4eff9c6c8baabdab58d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:27.000249 28513 tablet_service.cc:1511] Processing CreateTablet for tablet 8f7b66652f464893ad812d27e883a9a4 (DEFAULT_TABLE table=default.Table_697444611a2643bfb1fe30fc545c47f3 [id=4aece4c39f65473494e823b598e81bd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:27.002086 28800 ts_tablet_manager.cc:1916] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:27.005144 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet ac58e939730c4af681c2f08bcfbb6602 (table default.Table_ceb33d60d86e4fd898ae67c78da94a55 [id=3251ae257e0f49b99352dcc1ef004588]) successfully deleted
W20260618 13:59:27.005609 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 827cc0987bd541759e334fb620f04ed5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 827cc0987bd541759e334fb620f04ed5 already in progress: creating tablet
I20260618 13:59:27.005980 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet da4d4f03911f4ac099ecc2fa1d8af494 (table default.Table_99428babdde649b8b5d5fab801b58136 [id=f7cb92d4ab0742af9946f6bcf708df4d]) successfully deleted
W20260618 13:59:27.006354 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 02b5cc08ca0c4e719e0414b345211ba5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02b5cc08ca0c4e719e0414b345211ba5 already in progress: creating tablet
I20260618 13:59:27.006634 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 411b0c333ae7450f9ada90afee553615 (table default.Table_5d0d3ddc1f4a4badbb853268cb2bba65 [id=cd89063e7d294d3fbb3b58358e781c90]) successfully deleted
W20260618 13:59:27.006994 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 8b88ed7cf79c4929a8704f66691cbed4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b88ed7cf79c4929a8704f66691cbed4 already in progress: creating tablet
W20260618 13:59:27.007357 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 591465e2b57247a2b224cc8b09225811 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 591465e2b57247a2b224cc8b09225811 already in progress: creating tablet
I20260618 13:59:27.012601 28517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8af2bec820b842b4bf6b57482007bd6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:27.021945 28513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f7b66652f464893ad812d27e883a9a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:27.023036 28801 ts_tablet_manager.cc:1929] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:27.023665 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c01c8463055e4526b3c75190066f30b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:27.024224 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet b1f3dc9c3d874ca4ae42fd5e6d84fcb0 (table default.Table_ceb33d60d86e4fd898ae67c78da94a55 [id=3251ae257e0f49b99352dcc1ef004588]) successfully deleted
I20260618 13:59:27.024830 28512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b9f9649587b4c31951204b8bebe7ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:27.026412 28801 log.cc:1199] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/411b0c333ae7450f9ada90afee553615
I20260618 13:59:27.027189 28801 ts_tablet_manager.cc:1950] T 411b0c333ae7450f9ada90afee553615 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:27.028368 28801 tablet_replica.cc:333] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:27.028623 28801 raft_consensus.cc:2243] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:27.028930 28801 raft_consensus.cc:2272] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:27.029640 28801 ts_tablet_manager.cc:1916] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:27.030479 29185 consensus_queue.cc:1048] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:27.037727 28800 ts_tablet_manager.cc:1929] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:27.037914 28800 log.cc:1199] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/b0c9d4131ea24809ac081e7a73a21e1c
I20260618 13:59:27.040923 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 668c6c6f2034484ca5ccdd51c22dea1d (table default.Table_a6146ff7daf149b4bca36d8f09293605 [id=aa7194d6b3b2419294e5aa57d0043a11]) successfully deleted
W20260618 13:59:27.041554 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 827cc0987bd541759e334fb620f04ed5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 827cc0987bd541759e334fb620f04ed5 already in progress: creating tablet
I20260618 13:59:27.038388 28800 ts_tablet_manager.cc:1950] T b0c9d4131ea24809ac081e7a73a21e1c P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:27.053768 28224 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_571f6cccbfa3470a83b5542f128f2370"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:27.057996 29208 raft_consensus.cc:493] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.058224 29208 raft_consensus.cc:515] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.058919 29208 leader_election.cc:290] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:27.059279 28374 tablet_service.cc:1558] Processing DeleteTablet for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 with delete_type TABLET_DATA_DELETED (replaced by tablet cea4bc13eabc4dd98b4f96723498014c) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:27.059538 28802 tablet_replica.cc:333] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:27.059715 28802 raft_consensus.cc:2243] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:27.059878 28802 raft_consensus.cc:2272] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:27.060667 28802 ts_tablet_manager.cc:1916] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:27.065671 28227 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 3a0039a463ec4a9c82a21a3a69b565fc
I20260618 13:59:27.066398 28374 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a0039a463ec4a9c82a21a3a69b565fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:27.066690 28222 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 41107eb1d9164a9c98060003b4027412
I20260618 13:59:27.067030 28802 ts_tablet_manager.cc:1929] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:27.067152 28802 log.cc:1199] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/75b9c0754beb4e0e8ed3a7c1e9f368a9
I20260618 13:59:27.067490 28374 tablet_service.cc:1558] Processing DeleteTablet for tablet 41107eb1d9164a9c98060003b4027412 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f5d4296952e48a48756e6f27140da87) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:27.068032 28802 ts_tablet_manager.cc:1950] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:27.068240 28221 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet c7f42997416841a7b2c1ad347d2e9282
I20260618 13:59:27.069130 28374 tablet_service.cc:1558] Processing DeleteTablet for tablet c7f42997416841a7b2c1ad347d2e9282 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:27.069595 28226 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 75824a5d2f7e47838b60e6a2b785c6b2
W20260618 13:59:27.071062 28617 leader_election.cc:343] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: ba0d05ea44e1473fb4b24f61eab096e4
I20260618 13:59:27.071573 28228 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_3bd5981b65e545459eb539217e8e3599" } new_table_name: "default.Table_c99584ecc88948a097b39c22d68b3938" modify_external_catalogs: true
I20260618 13:59:27.074158 28229 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_dabd3b5bd3ff4784b839d502baec0c6e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:27.074795 28229 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dabd3b5bd3ff4784b839d502baec0c6e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:27.075202 28223 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_d6f2768172e8455882ce01051fc835a4" } new_table_name: "default.Table_e97ff5cd68d24da7a9a803e10b35dbe6" modify_external_catalogs: true
I20260618 13:59:27.076691 28222 catalog_manager.cc:6784] ReplaceTablet: tablet 41107eb1d9164a9c98060003b4027412 deleted and replaced by tablet 3f5d4296952e48a48756e6f27140da87
I20260618 13:59:27.077029 28222 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_69a1929cd67c4491bd769e49f6d4ab09" } new_table_name: "default.Table_4339ff084ae34b5eaad45284187a4d43" modify_external_catalogs: true
I20260618 13:59:27.078337 28220 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 7aa9f65d8a5c46fdb745ee17af946dce
I20260618 13:59:27.079236 28225 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet c3d523419ff244f085d6eac1a3231973
I20260618 13:59:27.079991 28225 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_f7398bc6e2f242d592730693cfb0c3b3" } new_table_name: "default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a" modify_external_catalogs: true
I20260618 13:59:27.082528 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 (table default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c]) successfully deleted
W20260618 13:59:27.083007 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 3a0039a463ec4a9c82a21a3a69b565fc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3a0039a463ec4a9c82a21a3a69b565fc already in progress: creating tablet
I20260618 13:59:27.083346 28374 tablet_service.cc:1558] Processing DeleteTablet for tablet 75824a5d2f7e47838b60e6a2b785c6b2 with delete_type TABLET_DATA_DELETED (replaced by tablet d4539f767c054e9e8ebc1f3f56108e7e) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:27.083614 28381 tablet_service.cc:1558] Processing DeleteTablet for tablet c3d523419ff244f085d6eac1a3231973 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:27.087121 28221 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_b583e9f1f2464ebe9df9b4663bb0a4f8" } modify_external_catalogs: true
I20260618 13:59:27.087627 28226 catalog_manager.cc:6784] ReplaceTablet: tablet 75824a5d2f7e47838b60e6a2b785c6b2 deleted and replaced by tablet d4539f767c054e9e8ebc1f3f56108e7e
I20260618 13:59:27.085382 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba0d05ea44e1473fb4b24f61eab096e4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:27.090696 28618 leader_election.cc:343] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:27.092559 28798 tablet_bootstrap.cc:492] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
W20260618 13:59:27.095551 28224 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_571f6cccbfa3470a83b5542f128f2370 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:27.096525 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 41107eb1d9164a9c98060003b4027412 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 41107eb1d9164a9c98060003b4027412 already in progress: creating tablet
W20260618 13:59:27.096832 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet c7f42997416841a7b2c1ad347d2e9282 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c7f42997416841a7b2c1ad347d2e9282 already in progress: creating tablet
I20260618 13:59:27.097860 28798 ts_tablet_manager.cc:1403] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.829s user 0.004s sys 0.032s
I20260618 13:59:27.097906 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 411b0c333ae7450f9ada90afee553615 (table default.Table_5d0d3ddc1f4a4badbb853268cb2bba65 [id=cd89063e7d294d3fbb3b58358e781c90]) successfully deleted
W20260618 13:59:27.098206 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 75824a5d2f7e47838b60e6a2b785c6b2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75824a5d2f7e47838b60e6a2b785c6b2 already in progress: creating tablet
W20260618 13:59:27.098405 28228 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 98b42d23832545ae81a7cdff19e4689f consensus state has no leader
W20260618 13:59:27.099112 28223 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f54910015987484e95b295352d859315 consensus state has no leader
I20260618 13:59:27.099570 28221 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_b583e9f1f2464ebe9df9b4663bb0a4f8" } modify_external_catalogs: true
I20260618 13:59:27.100957 28799 tablet_bootstrap.cc:492] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.101302 28799 ts_tablet_manager.cc:1403] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.233s user 0.032s sys 0.057s
I20260618 13:59:27.102208 28799 raft_consensus.cc:359] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.102527 28799 raft_consensus.cc:385] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.102686 28799 raft_consensus.cc:740] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.098246 28226 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_6f1f355b18a34ac09304992c96d40765"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:27.102874 28222 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1d8b3cfac0f648cd8b1c51102132c676 consensus state has no leader
W20260618 13:59:27.102988 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet c3d523419ff244f085d6eac1a3231973 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c3d523419ff244f085d6eac1a3231973 already in progress: creating tablet
I20260618 13:59:27.104444 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9559c5f2c9e94e858093dd2f0865c865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:27.107306 28228 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_c6e2f443374a47d2a787049499a0b168"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:27.107841 28228 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c6e2f443374a47d2a787049499a0b168 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:27.109412 28215 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.100.3:35503, user_credentials={real_user=slave}} blocked reactor thread for 55744.2us
W20260618 13:59:27.109957 28226 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f1f355b18a34ac09304992c96d40765 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:27.112126 28618 leader_election.cc:304] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:27.112807 28223 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_7ae70108b1714e5ca374cc653bed7b33"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:27.113472 28223 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7ae70108b1714e5ca374cc653bed7b33 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:27.114681 28220 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_5592094c6a024bd6ac90af5c18b0daab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:27.116992 28220 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5592094c6a024bd6ac90af5c18b0daab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:27.121335 28227 master_service.cc:960] ReplaceTablet: received request to replace tablet 283523e5b22843ba935eabe6283d3a99 from {username='slave'} at 127.0.0.1:45138
W20260618 13:59:27.124436 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet da4d4f03911f4ac099ecc2fa1d8af494 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da4d4f03911f4ac099ecc2fa1d8af494 already in progress: creating tablet
I20260618 13:59:27.125149 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 3752f49c9fc64071a45e0ce4ff4cfef2 (table default.Table_046616bed79749d5923cbd9ab9dff2b4 [id=af7cd415b54c4e4d8c7fb6f29dc8ff7b]) successfully deleted
W20260618 13:59:27.125587 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 02b5cc08ca0c4e719e0414b345211ba5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02b5cc08ca0c4e719e0414b345211ba5 already in progress: creating tablet
I20260618 13:59:27.126098 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet dca54111fea047d9ae0fabe376d89382 (table default.Table_77ced729210e46da993ea9e57db1d716 [id=fcccef3123f14001be3f111806a209a0]) successfully deleted
I20260618 13:59:27.133550 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 95db5ddaf5d5486988750efed5972eb6 (table default.Table_067483ebd76c49e595543cc25ff1db56 [id=5dadba9b472745259796cf9015e88c65]) successfully deleted
W20260618 13:59:27.134625 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 8b88ed7cf79c4929a8704f66691cbed4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b88ed7cf79c4929a8704f66691cbed4 already in progress: creating tablet
I20260618 13:59:27.135776 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 08a2dee3ac5a4bec96445a6001717eda (table default.Table_d81b926f20864cfe9915d642c53fa6f2 [id=80c0bb8d7847432783cb5c6c0bf76633]) successfully deleted
I20260618 13:59:27.136636 29208 raft_consensus.cc:2749] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:27.139357 28797 tablet_bootstrap.cc:492] T 0a51e830e3014726b1d1c2608973f036 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.139591 28797 ts_tablet_manager.cc:1403] T 0a51e830e3014726b1d1c2608973f036 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.148s user 0.028s sys 0.043s
I20260618 13:59:27.140537 28797 raft_consensus.cc:359] T 0a51e830e3014726b1d1c2608973f036 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.140753 28797 raft_consensus.cc:385] T 0a51e830e3014726b1d1c2608973f036 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.140856 28797 raft_consensus.cc:740] T 0a51e830e3014726b1d1c2608973f036 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.141127 28797 consensus_queue.cc:260] T 0a51e830e3014726b1d1c2608973f036 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.141599 28797 ts_tablet_manager.cc:1434] T 0a51e830e3014726b1d1c2608973f036 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:27.141950 28797 tablet_bootstrap.cc:492] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.144063 28797 tablet_bootstrap.cc:654] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.145481 28798 raft_consensus.cc:359] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.105458 28381 tablet_service.cc:1558] Processing DeleteTablet for tablet 7aa9f65d8a5c46fdb745ee17af946dce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:27.121661 28374 tablet_service.cc:1558] Processing DeleteTablet for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 with delete_type TABLET_DATA_DELETED (replaced by tablet cea4bc13eabc4dd98b4f96723498014c) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:27.121853 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet 37a6764a5e024903848561c9935b1a43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:27.146461 28799 consensus_queue.cc:260] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.147054 28802 tablet_replica.cc:333] T 37a6764a5e024903848561c9935b1a43 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:27.147279 28802 raft_consensus.cc:2243] T 37a6764a5e024903848561c9935b1a43 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:27.147449 28802 raft_consensus.cc:2272] T 37a6764a5e024903848561c9935b1a43 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:27.148797 28799 ts_tablet_manager.cc:1434] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.047s user 0.002s sys 0.000s
I20260618 13:59:27.149111 28799 tablet_bootstrap.cc:492] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.151191 28798 raft_consensus.cc:385] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.152307 28798 raft_consensus.cc:740] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.152529 28802 ts_tablet_manager.cc:1916] T 37a6764a5e024903848561c9935b1a43 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:27.154815 28798 consensus_queue.cc:260] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.156822 28799 tablet_bootstrap.cc:654] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.161545 28802 ts_tablet_manager.cc:1929] T 37a6764a5e024903848561c9935b1a43 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:27.161787 28802 log.cc:1199] T 37a6764a5e024903848561c9935b1a43 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/37a6764a5e024903848561c9935b1a43
I20260618 13:59:27.162210 28802 ts_tablet_manager.cc:1950] T 37a6764a5e024903848561c9935b1a43 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:27.167498 28799 tablet_bootstrap.cc:492] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.167717 28799 ts_tablet_manager.cc:1403] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.001s
I20260618 13:59:27.171044 28797 tablet_bootstrap.cc:492] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.171298 28797 ts_tablet_manager.cc:1403] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.029s user 0.004s sys 0.000s
I20260618 13:59:27.172245 28797 raft_consensus.cc:359] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.172564 28797 raft_consensus.cc:385] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.172698 28797 raft_consensus.cc:740] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.173120 28799 raft_consensus.cc:359] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.173116 28797 consensus_queue.cc:260] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.173331 28799 raft_consensus.cc:385] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.173403 28799 raft_consensus.cc:740] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.173956 28797 ts_tablet_manager.cc:1434] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:27.174357 28797 tablet_bootstrap.cc:492] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.175829 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet b0c9d4131ea24809ac081e7a73a21e1c (table default.Table_ddcc7a2b28ca4095be58d4c859fd964a [id=0bd492e029e346f68f2db501d798ace1]) successfully deleted
I20260618 13:59:27.176908 28797 tablet_bootstrap.cc:654] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.187660 28224 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_12d6cc95071949c6a7a297cd08b5f898" } new_table_name: "default.Table_f44be481def649db8daed0bf151a258c" modify_external_catalogs: true
I20260618 13:59:27.187987 28799 consensus_queue.cc:260] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.189535 28799 ts_tablet_manager.cc:1434] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.022s user 0.001s sys 0.001s
I20260618 13:59:27.189841 28799 tablet_bootstrap.cc:492] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.191020 28229 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_539ac216b6f4478eb994218d7d7980a4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:27.191622 28229 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_539ac216b6f4478eb994218d7d7980a4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:27.194324 28799 tablet_bootstrap.cc:654] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:27.195664 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 7aa9f65d8a5c46fdb745ee17af946dce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7aa9f65d8a5c46fdb745ee17af946dce already in progress: creating tablet
W20260618 13:59:27.195931 28214 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 because the tablet was not found. No further retry: Not found: Tablet not found: 75b9c0754beb4e0e8ed3a7c1e9f368a9
I20260618 13:59:27.196775 28214 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 37a6764a5e024903848561c9935b1a43 (table default.Table_88951dd073b74c99aa4a0bd7dcf2e19d [id=51cf184bc7634141841001c082262441]) successfully deleted
I20260618 13:59:27.197579 28384 tablet_service.cc:1511] Processing CreateTablet for tablet d0bcff62290c43c9be7cc496fbff6142 (DEFAULT_TABLE table=default.Table_bfcd14e3d49e4de09270ed85ec9cde4c [id=aa0452c5d31844c4b99cd207c0163445]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:27.197955 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0bcff62290c43c9be7cc496fbff6142. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:27.198400 28225 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 39b06ae7444a48dbbec9e096fd7e4052 consensus state has no leader
I20260618 13:59:27.199225 28797 tablet_bootstrap.cc:492] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.199481 28797 ts_tablet_manager.cc:1403] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.004s
I20260618 13:59:27.200460 28797 raft_consensus.cc:359] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.200786 28797 raft_consensus.cc:385] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.200943 28797 raft_consensus.cc:740] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.201321 28797 consensus_queue.cc:260] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.202083 28797 ts_tablet_manager.cc:1434] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:27.203912 28221 catalog_manager.cc:5983] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Sending DeleteTablet for 3 replicas of tablet 7572d17724d249739be1a1cb9468f909
W20260618 13:59:27.205380 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 591465e2b57247a2b224cc8b09225811 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 591465e2b57247a2b224cc8b09225811 already in progress: creating tablet
I20260618 13:59:27.206784 28374 tablet_service.cc:1511] Processing CreateTablet for tablet f695a7e6a925470787a708c365e6b86b (DEFAULT_TABLE table=default.Table_9763de0aea1d44d785311a303abb1d27 [id=34c58ef63c21476685cbc59915bbfd0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:27.207216 28374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f695a7e6a925470787a708c365e6b86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:27.210654 28381 tablet_service.cc:1558] Processing DeleteTablet for tablet 7572d17724d249739be1a1cb9468f909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:27 UTC) from {username='slave'} at 127.0.0.1:59166
I20260618 13:59:27.212720 28373 tablet_service.cc:1511] Processing CreateTablet for tablet 98446f40214c4ac984fcfe98310005c5 (DEFAULT_TABLE table=default.Table_83d021ed148243ffb453cb5e2484b7b4 [id=b3cae943ecdf410ca1cf711c5ddfe14a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:27.212946 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet ba0d05ea44e1473fb4b24f61eab096e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:27.213189 28373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98446f40214c4ac984fcfe98310005c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:27.213189 28659 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a0039a463ec4a9c82a21a3a69b565fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:27.213626 28663 tablet_service.cc:1558] Processing DeleteTablet for tablet 41107eb1d9164a9c98060003b4027412 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f5d4296952e48a48756e6f27140da87) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:27.213881 28649 tablet_service.cc:1558] Processing DeleteTablet for tablet c7f42997416841a7b2c1ad347d2e9282 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:27.214093 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 75824a5d2f7e47838b60e6a2b785c6b2 with delete_type TABLET_DATA_DELETED (replaced by tablet d4539f767c054e9e8ebc1f3f56108e7e) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:27.214282 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 68b8293a7c924a449307ab88a1cf9290 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:27.216243 28376 tablet_service.cc:1511] Processing CreateTablet for tablet 962bf304a285412fb2a529db5b994e9e (DEFAULT_TABLE table=default.Table_0fbace1c1aa746a6a2a57c878631218b [id=18b5971926ed4fff9048003a51d3a4fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:27.216435 28215 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 411b0c333ae7450f9ada90afee553615 (table default.Table_5d0d3ddc1f4a4badbb853268cb2bba65 [id=cd89063e7d294d3fbb3b58358e781c90]) successfully deleted
I20260618 13:59:27.216611 28376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 962bf304a285412fb2a529db5b994e9e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:27.216781 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 02b5cc08ca0c4e719e0414b345211ba5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02b5cc08ca0c4e719e0414b345211ba5 already in progress: creating tablet
W20260618 13:59:27.217003 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 591465e2b57247a2b224cc8b09225811 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 591465e2b57247a2b224cc8b09225811 already in progress: creating tablet
I20260618 13:59:27.217211 28222 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_152a483501b54a67bbdc0cae4959d452"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:27.219504 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet 8be4374ae7cf4907940ae90299bf9ba0 with delete_type TABLET_DATA_DELETED (replaced by tablet 092a9aba9c6e48f5be9cf5d55ba93678) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:27.219717 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet e025b64380314546af8a1ab90c817825 with delete_type TABLET_DATA_DELETED (replaced by tablet ece7909d056c4ba8a5dca6423f0c1781) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:27.219892 28654 tablet_service.cc:1558] Processing DeleteTablet for tablet fca76774f119435899188313a33dad98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:27.221210 28799 tablet_bootstrap.cc:492] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.221366 28799 ts_tablet_manager.cc:1403] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.032s user 0.004s sys 0.002s
I20260618 13:59:27.222251 28799 raft_consensus.cc:359] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.222579 28799 raft_consensus.cc:385] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.222558 28654 tablet_service.cc:1558] Processing DeleteTablet for tablet 7aa9f65d8a5c46fdb745ee17af946dce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:27.222687 28799 raft_consensus.cc:740] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.222939 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 7572d17724d249739be1a1cb9468f909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:27 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:27.222951 28799 consensus_queue.cc:260] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.223153 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet c3d523419ff244f085d6eac1a3231973 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:27.223575 28662 tablet_service.cc:1511] Processing CreateTablet for tablet d0bcff62290c43c9be7cc496fbff6142 (DEFAULT_TABLE table=default.Table_bfcd14e3d49e4de09270ed85ec9cde4c [id=aa0452c5d31844c4b99cd207c0163445]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:27.224678 28799 ts_tablet_manager.cc:1434] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260618 13:59:27.224965 28799 tablet_bootstrap.cc:492] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.226621 28799 tablet_bootstrap.cc:654] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.226612 28657 tablet_service.cc:1511] Processing CreateTablet for tablet f695a7e6a925470787a708c365e6b86b (DEFAULT_TABLE table=default.Table_9763de0aea1d44d785311a303abb1d27 [id=34c58ef63c21476685cbc59915bbfd0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:27.226853 29208 raft_consensus.cc:493] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.227034 29208 raft_consensus.cc:515] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.227727 29208 leader_election.cc:290] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:27.228806 28048 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu pid 28195
W20260618 13:59:27.230086 28617 leader_election.cc:343] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 68b8293a7c924a449307ab88a1cf9290
W20260618 13:59:27.230310 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 8b88ed7cf79c4929a8704f66691cbed4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b88ed7cf79c4929a8704f66691cbed4 already in progress: creating tablet
W20260618 13:59:27.230433 28214 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 7572d17724d249739be1a1cb9468f909 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7572d17724d249739be1a1cb9468f909 already in progress: creating tablet
I20260618 13:59:27.234754 28653 tablet_service.cc:1511] Processing CreateTablet for tablet 98446f40214c4ac984fcfe98310005c5 (DEFAULT_TABLE table=default.Table_83d021ed148243ffb453cb5e2484b7b4 [id=b3cae943ecdf410ca1cf711c5ddfe14a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:27.235267 28655 tablet_service.cc:1511] Processing CreateTablet for tablet 962bf304a285412fb2a529db5b994e9e (DEFAULT_TABLE table=default.Table_0fbace1c1aa746a6a2a57c878631218b [id=18b5971926ed4fff9048003a51d3a4fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:27.236279 28259 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 13:59:27.234323 28221 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_473a0c23203b404ea8e5ee0aed201e7b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:27.239789 28654 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a51e830e3014726b1d1c2608973f036 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:27.244547 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f695a7e6a925470787a708c365e6b86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:27.244998 28800 tablet_replica.cc:333] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:27.245193 28800 raft_consensus.cc:2243] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:27.245401 28800 raft_consensus.cc:2272] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:27.246606 28800 ts_tablet_manager.cc:1916] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:27.249752 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0bcff62290c43c9be7cc496fbff6142. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:27.250790 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98446f40214c4ac984fcfe98310005c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:27.252848 28800 ts_tablet_manager.cc:1929] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:27.252951 28800 log.cc:1199] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/ba0d05ea44e1473fb4b24f61eab096e4
I20260618 13:59:27.253291 28800 ts_tablet_manager.cc:1950] T ba0d05ea44e1473fb4b24f61eab096e4 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:27.254206 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet ba0d05ea44e1473fb4b24f61eab096e4 (table default.Table_e9839256914f48e6bb6480c5db7eb379 [id=7d40d815ea4544aea591ca23b96b5dd1]) successfully deleted
I20260618 13:59:27.254865 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 962bf304a285412fb2a529db5b994e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:27.260949 28800 tablet_replica.cc:333] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:27.261179 28800 raft_consensus.cc:2243] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:27.261335 28800 raft_consensus.cc:2272] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:27.262866 28800 ts_tablet_manager.cc:1916] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:27.268918 28800 ts_tablet_manager.cc:1929] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:27.269054 28800 log.cc:1199] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/68b8293a7c924a449307ab88a1cf9290
I20260618 13:59:27.269410 28800 ts_tablet_manager.cc:1950] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:27.278201 28800 tablet_replica.cc:333] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:27.278406 28800 raft_consensus.cc:2243] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:27.278692 28800 pending_rounds.cc:70] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257: Trying to abort 1 pending ops.
I20260618 13:59:27.278764 28800 pending_rounds.cc:77] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298216619847876608 op_type: NO_OP noop_request { }
I20260618 13:59:27.278895 28800 raft_consensus.cc:2889] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:27.278971 28797 tablet_bootstrap.cc:492] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.279023 28800 raft_consensus.cc:2272] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:27.282330 28800 ts_tablet_manager.cc:1916] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:27.283917 28797 tablet_bootstrap.cc:654] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.289641 28799 tablet_bootstrap.cc:492] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.289841 28799 ts_tablet_manager.cc:1403] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.065s user 0.000s sys 0.007s
I20260618 13:59:27.291313 28799 raft_consensus.cc:359] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.291548 28799 raft_consensus.cc:385] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.291630 28799 raft_consensus.cc:740] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.291894 28799 consensus_queue.cc:260] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.292479 28799 ts_tablet_manager.cc:1434] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 13:59:27.292856 28799 tablet_bootstrap.cc:492] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.294883 28799 tablet_bootstrap.cc:654] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.304530 28799 tablet_bootstrap.cc:492] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.304683 28799 ts_tablet_manager.cc:1403] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.005s
I20260618 13:59:27.306203 28799 raft_consensus.cc:359] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.306389 28799 raft_consensus.cc:385] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.306520 28799 raft_consensus.cc:740] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.306821 28799 consensus_queue.cc:260] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.310986 28799 ts_tablet_manager.cc:1434] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260618 13:59:27.311625 28797 tablet_bootstrap.cc:492] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.311985 28799 tablet_bootstrap.cc:492] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.313889 28799 tablet_bootstrap.cc:654] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.314246 28797 ts_tablet_manager.cc:1403] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.034s user 0.004s sys 0.001s
I20260618 13:59:27.315363 29208 raft_consensus.cc:493] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.315526 29208 raft_consensus.cc:515] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.315733 28797 raft_consensus.cc:359] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.315891 28797 raft_consensus.cc:385] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.315963 28797 raft_consensus.cc:740] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.316220 28797 consensus_queue.cc:260] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.317564 28797 ts_tablet_manager.cc:1434] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260618 13:59:27.318009 28797 tablet_bootstrap.cc:492] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.315371 28800 ts_tablet_manager.cc:1929] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:27.318572 28800 log.cc:1199] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/8be4374ae7cf4907940ae90299bf9ba0
I20260618 13:59:27.319060 28800 ts_tablet_manager.cc:1950] T 8be4374ae7cf4907940ae90299bf9ba0 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:27.321693 28797 tablet_bootstrap.cc:654] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.321913 28800 tablet_replica.cc:333] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:27.322966 28800 raft_consensus.cc:2243] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:27.323158 28800 raft_consensus.cc:2272] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:27.323685 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e025b64380314546af8a1ab90c817825" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:27.324368 28617 leader_election.cc:343] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: e025b64380314546af8a1ab90c817825
I20260618 13:59:27.327631 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68b8293a7c924a449307ab88a1cf9290" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:27.329996 28525 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a0039a463ec4a9c82a21a3a69b565fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:34092
W20260618 13:59:27.330749 28618 leader_election.cc:343] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:27.330919 28618 leader_election.cc:304] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
W20260618 13:59:27.331136 28618 raft_consensus.cc:2686] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260618 13:59:27.331333 28618 leader_election.cc:343] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:27.331442 28618 leader_election.cc:304] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
W20260618 13:59:27.331645 28618 raft_consensus.cc:2686] T 68b8293a7c924a449307ab88a1cf9290 P d6a169b16ced46ca9cfbc631709bb257: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260618 13:59:27.333005 28682 raft_consensus.cc:3060] T bab50e1f17a74eee8cad942525002996 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:27.333379 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet 75824a5d2f7e47838b60e6a2b785c6b2 with delete_type TABLET_DATA_DELETED (replaced by tablet d4539f767c054e9e8ebc1f3f56108e7e) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:27.333539 28511 tablet_service.cc:1558] Processing DeleteTablet for tablet c3d523419ff244f085d6eac1a3231973 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:27.333918 28516 tablet_service.cc:1558] Processing DeleteTablet for tablet 7aa9f65d8a5c46fdb745ee17af946dce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:27.334003 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet c7f42997416841a7b2c1ad347d2e9282 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:27.338917 28511 tablet_service.cc:1511] Processing CreateTablet for tablet 962bf304a285412fb2a529db5b994e9e (DEFAULT_TABLE table=default.Table_0fbace1c1aa746a6a2a57c878631218b [id=18b5971926ed4fff9048003a51d3a4fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:27.339839 28511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 962bf304a285412fb2a529db5b994e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:27.340759 28516 tablet_service.cc:1511] Processing CreateTablet for tablet 98446f40214c4ac984fcfe98310005c5 (DEFAULT_TABLE table=default.Table_83d021ed148243ffb453cb5e2484b7b4 [id=b3cae943ecdf410ca1cf711c5ddfe14a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:27.341539 28516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98446f40214c4ac984fcfe98310005c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:27.341621 29208 leader_election.cc:290] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:27.342819 28800 ts_tablet_manager.cc:1916] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:27.345732 28514 tablet_service.cc:1511] Processing CreateTablet for tablet f695a7e6a925470787a708c365e6b86b (DEFAULT_TABLE table=default.Table_9763de0aea1d44d785311a303abb1d27 [id=34c58ef63c21476685cbc59915bbfd0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:27.346048 28514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f695a7e6a925470787a708c365e6b86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:27.351061 29220 consensus_queue.cc:1048] T bab50e1f17a74eee8cad942525002996 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:27.351954 28524 tablet_service.cc:1511] Processing CreateTablet for tablet d0bcff62290c43c9be7cc496fbff6142 (DEFAULT_TABLE table=default.Table_bfcd14e3d49e4de09270ed85ec9cde4c [id=aa0452c5d31844c4b99cd207c0163445]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:27.352342 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0bcff62290c43c9be7cc496fbff6142. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:27.354645 28513 tablet_service.cc:1558] Processing DeleteTablet for tablet 41107eb1d9164a9c98060003b4027412 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f5d4296952e48a48756e6f27140da87) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:27.354842 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet 7572d17724d249739be1a1cb9468f909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:27 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:27.361105 28800 ts_tablet_manager.cc:1929] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:27.361258 28800 log.cc:1199] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/e025b64380314546af8a1ab90c817825
I20260618 13:59:27.361654 28800 ts_tablet_manager.cc:1950] T e025b64380314546af8a1ab90c817825 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:27.363071 28801 ts_tablet_manager.cc:1929] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:27.363178 28801 log.cc:1199] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/8be4374ae7cf4907940ae90299bf9ba0
I20260618 13:59:27.364274 28801 ts_tablet_manager.cc:1950] T 8be4374ae7cf4907940ae90299bf9ba0 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:27.377514 29183 raft_consensus.cc:493] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.377733 29183 raft_consensus.cc:515] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.378607 29183 leader_election.cc:290] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:27.379360 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c17e76ae9b44207998a43e3576908c4" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:27.379726 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c17e76ae9b44207998a43e3576908c4" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:27.380249 28346 leader_election.cc:343] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:27.380774 28346 leader_election.cc:343] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:27.380924 28346 leader_election.cc:304] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:27.381520 29183 raft_consensus.cc:2749] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:27.391140 28800 tablet_replica.cc:333] T fca76774f119435899188313a33dad98 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:27.391547 28800 raft_consensus.cc:2243] T fca76774f119435899188313a33dad98 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:27.391929 28800 raft_consensus.cc:2272] T fca76774f119435899188313a33dad98 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:27.392112 29185 raft_consensus.cc:493] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.392392 29185 raft_consensus.cc:515] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.395387 28800 ts_tablet_manager.cc:1916] T fca76774f119435899188313a33dad98 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:27.398732 29185 leader_election.cc:290] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:27.404421 28484 leader_election.cc:343] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 08a2dee3ac5a4bec96445a6001717eda
W20260618 13:59:27.404711 28483 leader_election.cc:343] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 08a2dee3ac5a4bec96445a6001717eda
I20260618 13:59:27.405054 28483 leader_election.cc:304] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:27.405671 29185 raft_consensus.cc:2749] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:27.405877 28798 ts_tablet_manager.cc:1434] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.295s user 0.040s sys 0.048s
I20260618 13:59:27.409370 28798 tablet_bootstrap.cc:492] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:27.419620 28799 tablet_bootstrap.cc:492] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.419811 28799 ts_tablet_manager.cc:1403] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.108s user 0.010s sys 0.040s
I20260618 13:59:27.420606 28799 raft_consensus.cc:359] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.420858 28799 raft_consensus.cc:385] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.420934 28799 raft_consensus.cc:740] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.421165 28799 consensus_queue.cc:260] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.421662 28799 ts_tablet_manager.cc:1434] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:27.422003 28799 tablet_bootstrap.cc:492] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.423875 28799 tablet_bootstrap.cc:654] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.425536 28797 tablet_bootstrap.cc:492] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.425678 28797 ts_tablet_manager.cc:1403] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.108s user 0.014s sys 0.036s
I20260618 13:59:27.426462 28797 raft_consensus.cc:359] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.426620 28797 raft_consensus.cc:385] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.426728 28797 raft_consensus.cc:740] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.426966 28797 consensus_queue.cc:260] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.427471 28797 ts_tablet_manager.cc:1434] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 13:59:27.427768 28797 tablet_bootstrap.cc:492] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.429780 28797 tablet_bootstrap.cc:654] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.432363 28799 tablet_bootstrap.cc:492] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.432516 28799 ts_tablet_manager.cc:1403] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260618 13:59:27.433384 28799 raft_consensus.cc:359] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.433593 28799 raft_consensus.cc:385] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.433707 28799 raft_consensus.cc:740] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.433934 28799 consensus_queue.cc:260] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.434459 28799 ts_tablet_manager.cc:1434] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:27.434803 28799 tablet_bootstrap.cc:492] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.436851 28799 tablet_bootstrap.cc:654] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.444685 28798 tablet_bootstrap.cc:654] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.445247 28799 tablet_bootstrap.cc:492] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.446086 28799 ts_tablet_manager.cc:1403] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.005s
I20260618 13:59:27.446936 28799 raft_consensus.cc:359] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.447161 28799 raft_consensus.cc:385] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.447247 28799 raft_consensus.cc:740] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.448951 28799 consensus_queue.cc:260] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.449467 28799 ts_tablet_manager.cc:1434] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260618 13:59:27.451030 28799 tablet_bootstrap.cc:492] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.455757 28799 tablet_bootstrap.cc:654] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.456517 29183 raft_consensus.cc:493] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.456740 29183 raft_consensus.cc:515] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.457469 29183 leader_election.cc:290] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:27.458271 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233040a16b0d4b61b977ca57c1afec2d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:27.458501 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233040a16b0d4b61b977ca57c1afec2d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:27.458928 28346 leader_election.cc:343] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:27.459173 28346 leader_election.cc:343] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:27.459276 28346 leader_election.cc:304] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:27.459540 29183 raft_consensus.cc:2749] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:27.460114 28800 ts_tablet_manager.cc:1929] T fca76774f119435899188313a33dad98 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:27.460340 28800 log.cc:1199] T fca76774f119435899188313a33dad98 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/fca76774f119435899188313a33dad98
I20260618 13:59:27.461099 28800 ts_tablet_manager.cc:1950] T fca76774f119435899188313a33dad98 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:27.462114 29183 raft_consensus.cc:493] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.462298 29183 raft_consensus.cc:515] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.462915 29183 leader_election.cc:290] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:27.463621 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "092a9aba9c6e48f5be9cf5d55ba93678" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:27.463634 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "092a9aba9c6e48f5be9cf5d55ba93678" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:27.464180 28346 leader_election.cc:343] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:27.464372 28346 leader_election.cc:343] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:27.464483 28346 leader_election.cc:304] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:27.464660 28799 tablet_bootstrap.cc:492] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.464794 28799 ts_tablet_manager.cc:1403] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260618 13:59:27.464893 29183 raft_consensus.cc:2749] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:27.467133 28799 raft_consensus.cc:359] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.467303 28799 raft_consensus.cc:385] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.467386 28799 raft_consensus.cc:740] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.467983 28799 consensus_queue.cc:260] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.468533 28799 ts_tablet_manager.cc:1434] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260618 13:59:27.469594 28799 tablet_bootstrap.cc:492] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.472935 28799 tablet_bootstrap.cc:654] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.473018 28800 tablet_replica.cc:333] T 0a51e830e3014726b1d1c2608973f036 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:27.473238 28800 raft_consensus.cc:2243] T 0a51e830e3014726b1d1c2608973f036 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:27.473840 28800 raft_consensus.cc:2272] T 0a51e830e3014726b1d1c2608973f036 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:27.477347 28800 ts_tablet_manager.cc:1916] T 0a51e830e3014726b1d1c2608973f036 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:27.480245 28799 tablet_bootstrap.cc:492] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.480436 28799 ts_tablet_manager.cc:1403] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.004s
I20260618 13:59:27.481314 28799 raft_consensus.cc:359] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.481555 28799 raft_consensus.cc:385] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.481659 28799 raft_consensus.cc:740] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.481968 28799 consensus_queue.cc:260] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.485329 28799 ts_tablet_manager.cc:1434] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260618 13:59:27.485689 28799 tablet_bootstrap.cc:492] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.488817 28799 tablet_bootstrap.cc:654] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.495419 28799 tablet_bootstrap.cc:492] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.495592 28799 ts_tablet_manager.cc:1403] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.002s
I20260618 13:59:27.496416 28799 raft_consensus.cc:359] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.496611 28799 raft_consensus.cc:385] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.496718 28799 raft_consensus.cc:740] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.496940 28799 consensus_queue.cc:260] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.497418 28799 ts_tablet_manager.cc:1434] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:27.498273 28797 tablet_bootstrap.cc:492] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.498737 28797 ts_tablet_manager.cc:1403] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.071s user 0.005s sys 0.026s
I20260618 13:59:27.499601 28797 raft_consensus.cc:359] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.499980 28797 raft_consensus.cc:385] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.500100 28797 raft_consensus.cc:740] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.497051 28798 tablet_bootstrap.cc:492] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:27.500504 28797 consensus_queue.cc:260] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.501243 28798 ts_tablet_manager.cc:1403] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.091s user 0.002s sys 0.002s
I20260618 13:59:27.501392 28797 ts_tablet_manager.cc:1434] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 13:59:27.501820 28799 tablet_bootstrap.cc:492] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.502166 28797 tablet_bootstrap.cc:492] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.504127 28797 tablet_bootstrap.cc:654] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.506340 28800 ts_tablet_manager.cc:1929] T 0a51e830e3014726b1d1c2608973f036 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:27.506522 28800 log.cc:1199] T 0a51e830e3014726b1d1c2608973f036 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/0a51e830e3014726b1d1c2608973f036
I20260618 13:59:27.507043 28800 ts_tablet_manager.cc:1950] T 0a51e830e3014726b1d1c2608973f036 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:27.516893 28799 tablet_bootstrap.cc:654] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.525543 28799 tablet_bootstrap.cc:492] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.525730 28799 ts_tablet_manager.cc:1403] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.005s
I20260618 13:59:27.526319 28797 tablet_bootstrap.cc:492] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.526535 28797 ts_tablet_manager.cc:1403] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.024s user 0.012s sys 0.004s
I20260618 13:59:27.526600 28799 raft_consensus.cc:359] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.526831 28799 raft_consensus.cc:385] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.526918 28799 raft_consensus.cc:740] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.527215 28799 consensus_queue.cc:260] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.527611 28797 raft_consensus.cc:359] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.527833 28797 raft_consensus.cc:385] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.527953 28797 raft_consensus.cc:740] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.528214 28797 consensus_queue.cc:260] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.528741 28797 ts_tablet_manager.cc:1434] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:27.529224 28797 tablet_bootstrap.cc:492] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.531062 28797 tablet_bootstrap.cc:654] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.536312 28799 ts_tablet_manager.cc:1434] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260618 13:59:27.538800 28798 raft_consensus.cc:359] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.539223 28798 raft_consensus.cc:385] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.539331 28798 raft_consensus.cc:740] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.540108 28798 consensus_queue.cc:260] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.540812 28799 tablet_bootstrap.cc:492] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.543537 28799 tablet_bootstrap.cc:654] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.543561 28798 ts_tablet_manager.cc:1434] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.042s user 0.002s sys 0.000s
I20260618 13:59:27.545495 28798 tablet_bootstrap.cc:492] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:27.547169 28798 tablet_bootstrap.cc:654] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.548735 28797 tablet_bootstrap.cc:492] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.549018 28797 ts_tablet_manager.cc:1403] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.006s
I20260618 13:59:27.550098 28797 raft_consensus.cc:359] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.550448 28797 raft_consensus.cc:385] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.550567 28797 raft_consensus.cc:740] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.550899 28797 consensus_queue.cc:260] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.551441 28797 ts_tablet_manager.cc:1434] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 13:59:27.552215 28799 tablet_bootstrap.cc:492] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.552348 28799 ts_tablet_manager.cc:1403] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.001s
I20260618 13:59:27.553031 28799 raft_consensus.cc:359] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.553179 28799 raft_consensus.cc:385] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.553263 28799 raft_consensus.cc:740] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.553618 28799 consensus_queue.cc:260] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.555405 28799 ts_tablet_manager.cc:1434] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:27.555673 28799 tablet_bootstrap.cc:492] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.559404 28799 tablet_bootstrap.cc:654] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.561107 28798 tablet_bootstrap.cc:492] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:27.561252 28798 ts_tablet_manager.cc:1403] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.004s
I20260618 13:59:27.562470 28798 raft_consensus.cc:359] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.562645 28798 raft_consensus.cc:385] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.562727 28798 raft_consensus.cc:740] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.562963 28798 consensus_queue.cc:260] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.564515 28798 ts_tablet_manager.cc:1434] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260618 13:59:27.564779 28798 tablet_bootstrap.cc:492] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:27.565093 28797 tablet_bootstrap.cc:492] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.567306 28798 tablet_bootstrap.cc:654] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.567628 28799 tablet_bootstrap.cc:492] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.567796 28799 ts_tablet_manager.cc:1403] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20260618 13:59:27.568087 28797 tablet_bootstrap.cc:654] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.568713 28799 raft_consensus.cc:359] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.569430 28799 raft_consensus.cc:385] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.569530 28799 raft_consensus.cc:740] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.569820 28799 consensus_queue.cc:260] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.570367 28799 ts_tablet_manager.cc:1434] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:27.575515 28799 tablet_bootstrap.cc:492] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.577612 28799 tablet_bootstrap.cc:654] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.580973 29208 raft_consensus.cc:493] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.581137 29208 raft_consensus.cc:515] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.581734 29208 leader_election.cc:290] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:27.582828 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da4d4f03911f4ac099ecc2fa1d8af494" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:27.583663 28617 leader_election.cc:343] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: da4d4f03911f4ac099ecc2fa1d8af494
W20260618 13:59:27.584089 28618 leader_election.cc:343] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:27.584208 28618 leader_election.cc:304] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:27.584640 29208 raft_consensus.cc:2749] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:27.589046 28799 tablet_bootstrap.cc:492] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.589044 28797 tablet_bootstrap.cc:492] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.589196 28797 ts_tablet_manager.cc:1403] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.010s
I20260618 13:59:27.589249 28799 ts_tablet_manager.cc:1403] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.000s
I20260618 13:59:27.590026 28799 raft_consensus.cc:359] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.590193 28799 raft_consensus.cc:385] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.590287 28799 raft_consensus.cc:740] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.590026 28797 raft_consensus.cc:359] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.590520 28797 raft_consensus.cc:385] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.590548 28799 consensus_queue.cc:260] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.590653 28797 raft_consensus.cc:740] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.590945 28797 consensus_queue.cc:260] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.591542 28797 ts_tablet_manager.cc:1434] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:27.591905 28799 ts_tablet_manager.cc:1434] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:27.593470 28798 tablet_bootstrap.cc:492] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:27.593618 28798 ts_tablet_manager.cc:1403] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.015s
I20260618 13:59:27.592810 28797 tablet_bootstrap.cc:492] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.595621 28798 raft_consensus.cc:359] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.595813 28798 raft_consensus.cc:385] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.595870 28798 raft_consensus.cc:740] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.596104 28798 consensus_queue.cc:260] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.597334 28798 ts_tablet_manager.cc:1434] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 13:59:27.597992 28797 tablet_bootstrap.cc:654] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.598059 28798 tablet_bootstrap.cc:492] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:27.601063 28798 tablet_bootstrap.cc:654] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.609295 28799 tablet_bootstrap.cc:492] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.611598 28799 tablet_bootstrap.cc:654] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.613699 28797 tablet_bootstrap.cc:492] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.613863 28797 ts_tablet_manager.cc:1403] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.022s user 0.008s sys 0.000s
I20260618 13:59:27.614570 28797 raft_consensus.cc:359] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.614774 28797 raft_consensus.cc:385] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.614847 28797 raft_consensus.cc:740] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.615074 28797 consensus_queue.cc:260] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.619879 28797 ts_tablet_manager.cc:1434] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260618 13:59:27.620194 28797 tablet_bootstrap.cc:492] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.621747 28797 tablet_bootstrap.cc:654] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.628823 28797 tablet_bootstrap.cc:492] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.628968 28797 ts_tablet_manager.cc:1403] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260618 13:59:27.629676 28797 raft_consensus.cc:359] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.629763 28799 tablet_bootstrap.cc:492] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.629848 28797 raft_consensus.cc:385] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.629933 28797 raft_consensus.cc:740] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.629937 28799 ts_tablet_manager.cc:1403] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.004s
I20260618 13:59:27.630129 28797 consensus_queue.cc:260] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.630617 28797 ts_tablet_manager.cc:1434] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:27.630966 28797 tablet_bootstrap.cc:492] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.632491 28797 tablet_bootstrap.cc:654] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.639559 28799 raft_consensus.cc:359] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.639783 28799 raft_consensus.cc:385] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.639863 28799 raft_consensus.cc:740] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.640128 28799 consensus_queue.cc:260] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.640632 28799 ts_tablet_manager.cc:1434] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260618 13:59:27.640977 28799 tablet_bootstrap.cc:492] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.645438 28798 tablet_bootstrap.cc:492] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:27.645591 28798 ts_tablet_manager.cc:1403] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.048s user 0.017s sys 0.013s
I20260618 13:59:27.646392 28798 raft_consensus.cc:359] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.646593 28798 raft_consensus.cc:385] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.646675 28798 raft_consensus.cc:740] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.646971 28798 consensus_queue.cc:260] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.647768 28799 tablet_bootstrap.cc:654] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.647754 28798 ts_tablet_manager.cc:1434] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 13:59:27.650105 28798 tablet_bootstrap.cc:492] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:27.651721 28798 tablet_bootstrap.cc:654] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.652781 28797 tablet_bootstrap.cc:492] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.653199 28799 tablet_bootstrap.cc:492] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.653354 28797 ts_tablet_manager.cc:1403] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.000s
I20260618 13:59:27.653468 28799 ts_tablet_manager.cc:1403] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.003s
I20260618 13:59:27.654146 28797 raft_consensus.cc:359] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.654624 28797 raft_consensus.cc:385] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.654740 28797 raft_consensus.cc:740] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.654183 28799 raft_consensus.cc:359] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.655225 28797 consensus_queue.cc:260] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.655409 28799 raft_consensus.cc:385] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.655499 28799 raft_consensus.cc:740] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.655723 28797 ts_tablet_manager.cc:1434] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260618 13:59:27.655755 28799 consensus_queue.cc:260] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.656224 28799 ts_tablet_manager.cc:1434] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260618 13:59:27.656608 28799 tablet_bootstrap.cc:492] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.659919 28799 tablet_bootstrap.cc:654] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.663378 28798 tablet_bootstrap.cc:492] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:27.663925 28798 ts_tablet_manager.cc:1403] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.007s
I20260618 13:59:27.664039 28797 tablet_bootstrap.cc:492] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.665390 28798 raft_consensus.cc:359] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.665602 28797 tablet_bootstrap.cc:654] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.666008 28798 raft_consensus.cc:385] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.666097 28798 raft_consensus.cc:740] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.666332 28798 consensus_queue.cc:260] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.667711 28798 ts_tablet_manager.cc:1434] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:27.668004 28798 tablet_bootstrap.cc:492] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:27.670446 28798 tablet_bootstrap.cc:654] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.679984 28799 tablet_bootstrap.cc:492] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.680308 28799 ts_tablet_manager.cc:1403] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.003s
I20260618 13:59:27.681365 28799 raft_consensus.cc:359] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.681718 28799 raft_consensus.cc:385] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.681859 28799 raft_consensus.cc:740] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.682286 28799 consensus_queue.cc:260] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.689009 28799 ts_tablet_manager.cc:1434] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260618 13:59:27.694849 28797 tablet_bootstrap.cc:492] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.695051 28797 ts_tablet_manager.cc:1403] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.039s user 0.024s sys 0.006s
I20260618 13:59:27.695039 28799 tablet_bootstrap.cc:492] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.695333 29183 raft_consensus.cc:493] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.695519 29183 raft_consensus.cc:515] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.695984 28797 raft_consensus.cc:359] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.696213 28797 raft_consensus.cc:385] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.696259 29183 leader_election.cc:290] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:27.696395 28797 raft_consensus.cc:740] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.696880 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd2385e6752d49429422fc854539e9c6" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:27.697503 28346 leader_election.cc:343] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:27.696604 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd2385e6752d49429422fc854539e9c6" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:27.698112 28346 leader_election.cc:343] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:27.698239 28346 leader_election.cc:304] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:27.698570 29183 raft_consensus.cc:2749] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:27.697959 28797 consensus_queue.cc:260] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.699707 28799 tablet_bootstrap.cc:654] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.703224 29208 raft_consensus.cc:493] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.703372 29208 raft_consensus.cc:515] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.703888 29208 leader_election.cc:290] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:27.704443 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1a06078974e4bfbbc4b949802a39bf1" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:27.704617 28411 raft_consensus.cc:2468] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:27.704995 28617 leader_election.cc:304] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:27.705725 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1a06078974e4bfbbc4b949802a39bf1" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:27.706208 28618 leader_election.cc:343] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:27.706844 29208 raft_consensus.cc:2804] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:27.706961 29208 raft_consensus.cc:493] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:27.707083 29208 raft_consensus.cc:3060] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:27.707520 28798 tablet_bootstrap.cc:492] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:27.707654 28798 ts_tablet_manager.cc:1403] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.040s user 0.017s sys 0.019s
I20260618 13:59:27.708374 28798 raft_consensus.cc:359] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.708540 28798 raft_consensus.cc:385] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.709057 28798 raft_consensus.cc:740] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.709293 28798 consensus_queue.cc:260] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.709911 28798 ts_tablet_manager.cc:1434] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:27.711011 29208 raft_consensus.cc:515] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.712517 29208 leader_election.cc:290] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:27.712942 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1a06078974e4bfbbc4b949802a39bf1" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:27.713126 28411 raft_consensus.cc:3060] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:27.713176 28797 ts_tablet_manager.cc:1434] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.018s user 0.001s sys 0.000s
I20260618 13:59:27.713598 28797 tablet_bootstrap.cc:492] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.715858 28411 raft_consensus.cc:2468] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:27.715936 28798 tablet_bootstrap.cc:492] T a1a06078974e4bfbbc4b949802a39bf1 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:27.716305 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1a06078974e4bfbbc4b949802a39bf1" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:27.717258 28618 leader_election.cc:343] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:27.717638 28798 tablet_bootstrap.cc:654] T a1a06078974e4bfbbc4b949802a39bf1 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.720374 28617 leader_election.cc:304] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:27.723544 29208 raft_consensus.cc:2804] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:27.724385 29208 raft_consensus.cc:697] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:27.724829 29208 consensus_queue.cc:237] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.727558 28797 tablet_bootstrap.cc:654] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.730139 29208 raft_consensus.cc:493] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.730341 29208 raft_consensus.cc:515] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.730682 29266 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7637285570574a6dbcef1de6f84ecac0)
I20260618 13:59:27.731096 29208 leader_election.cc:290] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:27.731490 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eead369de58f40378254697cbbbc6958" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:27.732106 28617 leader_election.cc:343] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: eead369de58f40378254697cbbbc6958
W20260618 13:59:27.732300 28618 leader_election.cc:343] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:27.732484 28618 leader_election.cc:304] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:27.733019 29208 raft_consensus.cc:2749] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:27.730903 29266 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:27.735671 29266 leader_election.cc:290] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a950ab7cd66b4cb0b29592aa692c2dd0 (127.27.100.61:45967), 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083)
I20260618 13:59:27.736474 29048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 488 } } ignore_live_leader: false dest_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" is_pre_election: true
I20260618 13:59:27.740257 28797 tablet_bootstrap.cc:492] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.740563 28797 ts_tablet_manager.cc:1403] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.003s
I20260618 13:59:27.742177 28797 raft_consensus.cc:359] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.742551 28797 raft_consensus.cc:385] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.742708 28797 raft_consensus.cc:740] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.743129 28797 consensus_queue.cc:260] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.747388 28797 ts_tablet_manager.cc:1434] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260618 13:59:27.747936 28797 tablet_bootstrap.cc:492] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.753438 28797 tablet_bootstrap.cc:654] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.758252 28799 tablet_bootstrap.cc:492] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.758451 28799 ts_tablet_manager.cc:1403] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.064s user 0.022s sys 0.032s
I20260618 13:59:27.759423 28799 raft_consensus.cc:359] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.759639 28799 raft_consensus.cc:385] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.759732 28799 raft_consensus.cc:740] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.760046 28799 consensus_queue.cc:260] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.765514 29270 raft_consensus.cc:493] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7637285570574a6dbcef1de6f84ecac0)
I20260618 13:59:27.765851 29270 raft_consensus.cc:515] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:27.766156 28798 tablet_bootstrap.cc:492] T a1a06078974e4bfbbc4b949802a39bf1 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:27.766337 28798 ts_tablet_manager.cc:1403] T a1a06078974e4bfbbc4b949802a39bf1 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.050s user 0.015s sys 0.023s
I20260618 13:59:27.767220 28798 raft_consensus.cc:359] T a1a06078974e4bfbbc4b949802a39bf1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.767428 28798 raft_consensus.cc:385] T a1a06078974e4bfbbc4b949802a39bf1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.767505 28798 raft_consensus.cc:740] T a1a06078974e4bfbbc4b949802a39bf1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.767756 28799 ts_tablet_manager.cc:1434] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.009s user 0.000s sys 0.002s
I20260618 13:59:27.767807 28798 consensus_queue.cc:260] T a1a06078974e4bfbbc4b949802a39bf1 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.768366 28798 ts_tablet_manager.cc:1434] T a1a06078974e4bfbbc4b949802a39bf1 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260618 13:59:27.768779 29270 leader_election.cc:290] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e4d4f4566be42dfbafb7c0e40fb17ff (127.27.100.62:43001), 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083)
I20260618 13:59:27.769009 28096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 500 } } ignore_live_leader: false dest_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" is_pre_election: true
I20260618 13:59:27.769264 28798 tablet_bootstrap.cc:492] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:27.769243 28096 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a950ab7cd66b4cb0b29592aa692c2dd0 in term 1.
I20260618 13:59:27.769851 28989 leader_election.cc:304] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e4d4f4566be42dfbafb7c0e40fb17ff, a950ab7cd66b4cb0b29592aa692c2dd0; no voters:
I20260618 13:59:27.770164 29183 raft_consensus.cc:493] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.770325 29183 raft_consensus.cc:515] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.770525 29270 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260618 13:59:27.770656 29270 raft_consensus.cc:493] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7637285570574a6dbcef1de6f84ecac0)
I20260618 13:59:27.770752 29270 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 1 FOLLOWER]: Advancing to term 2
I20260618 13:59:27.770906 29183 leader_election.cc:290] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:27.771901 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "751cd9bd961b4f56a588ed02a05f0212" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:27.772518 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "751cd9bd961b4f56a588ed02a05f0212" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:27.773799 28346 leader_election.cc:343] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:27.773993 28346 leader_election.cc:343] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:27.774101 28346 leader_election.cc:304] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:27.774449 29183 raft_consensus.cc:2749] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:27.775310 29270 raft_consensus.cc:515] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:27.775502 28798 tablet_bootstrap.cc:654] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.775607 28799 tablet_bootstrap.cc:492] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.777065 28797 tablet_bootstrap.cc:492] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.777251 28797 ts_tablet_manager.cc:1403] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.029s user 0.013s sys 0.000s
I20260618 13:59:27.777360 28799 tablet_bootstrap.cc:654] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.778128 28797 raft_consensus.cc:359] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.778412 28797 raft_consensus.cc:385] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.778378 29230 raft_consensus.cc:493] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.778559 28797 raft_consensus.cc:740] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.778577 29230 raft_consensus.cc:515] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.778904 28797 consensus_queue.cc:260] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.779680 29230 leader_election.cc:290] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:27.781991 28484 leader_election.cc:343] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: ba0d05ea44e1473fb4b24f61eab096e4
I20260618 13:59:27.782377 28797 ts_tablet_manager.cc:1434] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
W20260618 13:59:27.782560 28483 leader_election.cc:343] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: ba0d05ea44e1473fb4b24f61eab096e4
I20260618 13:59:27.782724 28797 tablet_bootstrap.cc:492] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.782693 28483 leader_election.cc:304] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:27.783397 28799 tablet_bootstrap.cc:492] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.783557 28799 ts_tablet_manager.cc:1403] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260618 13:59:27.784399 28797 tablet_bootstrap.cc:654] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.784297 28799 raft_consensus.cc:359] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.784493 28799 raft_consensus.cc:385] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.784561 28799 raft_consensus.cc:740] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.785187 28799 consensus_queue.cc:260] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.791107 28799 ts_tablet_manager.cc:1434] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.007s user 0.001s sys 0.002s
I20260618 13:59:27.791400 28799 tablet_bootstrap.cc:492] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.792474 29230 raft_consensus.cc:2749] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:27.794582 29208 raft_consensus.cc:493] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.794732 29208 raft_consensus.cc:515] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.795231 28799 tablet_bootstrap.cc:654] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.795440 29208 leader_election.cc:290] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:27.797039 28617 leader_election.cc:343] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 4a01dfb4d59549399c94640f16045555
I20260618 13:59:27.797786 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a01dfb4d59549399c94640f16045555" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:27.798511 28618 leader_election.cc:343] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:27.798640 28618 leader_election.cc:304] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:27.799125 29270 leader_election.cc:290] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [CANDIDATE]: Term 2 election: Requested vote from peers 9e4d4f4566be42dfbafb7c0e40fb17ff (127.27.100.62:43001), 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083)
I20260618 13:59:27.799206 28798 tablet_bootstrap.cc:492] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:27.799360 28798 ts_tablet_manager.cc:1403] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.030s user 0.015s sys 0.005s
I20260618 13:59:27.799394 28096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 500 } } ignore_live_leader: false dest_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff"
I20260618 13:59:27.799471 29208 raft_consensus.cc:2749] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:27.799600 28096 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 1 FOLLOWER]: Advancing to term 2
I20260618 13:59:27.800186 28798 raft_consensus.cc:359] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.800374 28798 raft_consensus.cc:385] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.800465 28798 raft_consensus.cc:740] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.800743 28799 tablet_bootstrap.cc:492] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.800724 28798 consensus_queue.cc:260] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.800968 28799 ts_tablet_manager.cc:1403] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.002s
I20260618 13:59:27.801672 28798 ts_tablet_manager.cc:1434] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:27.801769 28797 tablet_bootstrap.cc:492] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.801800 28799 raft_consensus.cc:359] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.801973 28799 raft_consensus.cc:385] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.802068 28797 ts_tablet_manager.cc:1403] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.000s
I20260618 13:59:27.802057 28799 raft_consensus.cc:740] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.802333 28096 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a950ab7cd66b4cb0b29592aa692c2dd0 in term 2.
I20260618 13:59:27.802286 28799 consensus_queue.cc:260] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.802937 28799 ts_tablet_manager.cc:1434] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:27.803282 28799 tablet_bootstrap.cc:492] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.804421 28989 leader_election.cc:304] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e4d4f4566be42dfbafb7c0e40fb17ff, a950ab7cd66b4cb0b29592aa692c2dd0; no voters:
I20260618 13:59:27.804903 28797 raft_consensus.cc:359] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.805099 28797 raft_consensus.cc:385] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.805167 28797 raft_consensus.cc:740] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.805465 28798 tablet_bootstrap.cc:492] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:27.805478 28799 tablet_bootstrap.cc:654] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.807200 28798 tablet_bootstrap.cc:654] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.807654 28797 consensus_queue.cc:260] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.809325 28797 ts_tablet_manager.cc:1434] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20260618 13:59:27.809628 28797 tablet_bootstrap.cc:492] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.811735 28797 tablet_bootstrap.cc:654] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.817251 28798 tablet_bootstrap.cc:492] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:27.817409 28798 ts_tablet_manager.cc:1403] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.006s
I20260618 13:59:27.818086 28798 raft_consensus.cc:359] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.818269 28798 raft_consensus.cc:385] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.818327 28798 raft_consensus.cc:740] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.818641 28798 consensus_queue.cc:260] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.819226 28798 ts_tablet_manager.cc:1434] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:27.819725 28798 tablet_bootstrap.cc:492] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:27.821122 29230 raft_consensus.cc:493] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.821275 29230 raft_consensus.cc:515] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.821888 29230 leader_election.cc:290] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:27.823237 28484 leader_election.cc:343] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 68b8293a7c924a449307ab88a1cf9290
I20260618 13:59:27.823763 28797 tablet_bootstrap.cc:492] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.823916 28797 ts_tablet_manager.cc:1403] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.002s
W20260618 13:59:27.824180 28483 leader_election.cc:343] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 68b8293a7c924a449307ab88a1cf9290
I20260618 13:59:27.824306 28483 leader_election.cc:304] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:27.824625 29230 raft_consensus.cc:2749] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:27.824788 28797 raft_consensus.cc:359] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.824975 28797 raft_consensus.cc:385] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.825057 28797 raft_consensus.cc:740] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.825650 28798 tablet_bootstrap.cc:654] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.826161 28797 consensus_queue.cc:260] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.828409 28797 ts_tablet_manager.cc:1434] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 13:59:27.828676 28797 tablet_bootstrap.cc:492] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.830571 28797 tablet_bootstrap.cc:654] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.838735 28797 tablet_bootstrap.cc:492] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.838922 28797 ts_tablet_manager.cc:1403] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.004s
I20260618 13:59:27.839985 28797 raft_consensus.cc:359] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.840235 28797 raft_consensus.cc:385] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.840334 28797 raft_consensus.cc:740] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.840626 28797 consensus_queue.cc:260] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.843343 28797 ts_tablet_manager.cc:1434] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:27.843750 28797 tablet_bootstrap.cc:492] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.845999 28797 tablet_bootstrap.cc:654] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.855746 29183 raft_consensus.cc:493] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.855995 29183 raft_consensus.cc:515] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.856529 29270 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 2 FOLLOWER]: Leader election won for term 2
I20260618 13:59:27.856722 29183 leader_election.cc:290] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:27.857237 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f89c9954d48e407c96c294c26644e25c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:27.857331 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f89c9954d48e407c96c294c26644e25c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:27.857769 28346 leader_election.cc:343] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:27.857991 28346 leader_election.cc:343] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:27.858103 28346 leader_election.cc:304] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:27.858493 29183 raft_consensus.cc:2749] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:27.858944 28798 tablet_bootstrap.cc:492] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:27.859150 28798 ts_tablet_manager.cc:1403] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.040s user 0.016s sys 0.014s
I20260618 13:59:27.860010 28798 raft_consensus.cc:359] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.860257 28798 raft_consensus.cc:385] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.860389 28798 raft_consensus.cc:740] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.860684 28798 consensus_queue.cc:260] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.861479 28798 ts_tablet_manager.cc:1434] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:27.862206 28798 tablet_bootstrap.cc:492] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:27.864719 28798 tablet_bootstrap.cc:654] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.866007 28797 tablet_bootstrap.cc:492] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.866161 28797 ts_tablet_manager.cc:1403] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.005s
I20260618 13:59:27.867136 28797 raft_consensus.cc:359] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.867350 28797 raft_consensus.cc:385] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.867444 28797 raft_consensus.cc:740] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.867731 28797 consensus_queue.cc:260] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.870002 28797 ts_tablet_manager.cc:1434] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260618 13:59:27.870285 28797 tablet_bootstrap.cc:492] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.870682 28798 tablet_bootstrap.cc:492] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:27.870868 28798 ts_tablet_manager.cc:1403] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260618 13:59:27.870991 29230 raft_consensus.cc:493] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.871201 29230 raft_consensus.cc:515] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.871685 28798 raft_consensus.cc:359] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.871876 28798 raft_consensus.cc:385] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.871985 28797 tablet_bootstrap.cc:654] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.871958 28798 raft_consensus.cc:740] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.872018 29230 leader_election.cc:290] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:27.872282 28798 consensus_queue.cc:260] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.872871 28798 ts_tablet_manager.cc:1434] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:27.873143 28798 tablet_bootstrap.cc:492] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
W20260618 13:59:27.874233 28483 leader_election.cc:343] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: dca54111fea047d9ae0fabe376d89382
I20260618 13:59:27.874619 28798 tablet_bootstrap.cc:654] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:27.874788 28484 leader_election.cc:343] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: dca54111fea047d9ae0fabe376d89382
I20260618 13:59:27.875344 28484 leader_election.cc:304] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:27.875770 29230 raft_consensus.cc:2749] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:27.880143 28797 tablet_bootstrap.cc:492] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.880336 28797 ts_tablet_manager.cc:1403] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260618 13:59:27.880792 29270 raft_consensus.cc:697] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [term 2 LEADER]: Becoming Leader. State: Replica: a950ab7cd66b4cb0b29592aa692c2dd0, State: Running, Role: LEADER
I20260618 13:59:27.881556 29270 consensus_queue.cc:237] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 499, Committed index: 499, Last appended: 1.500, Last appended by leader: 500, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:27.883356 28797 raft_consensus.cc:359] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.883634 28797 raft_consensus.cc:385] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.883738 28797 raft_consensus.cc:740] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.884044 28797 consensus_queue.cc:260] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.884797 28797 ts_tablet_manager.cc:1434] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260618 13:59:27.885156 28797 tablet_bootstrap.cc:492] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.887056 29282 sys_catalog.cc:455] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a950ab7cd66b4cb0b29592aa692c2dd0. Latest consensus state: current_term: 2 leader_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:27.887405 29282 sys_catalog.cc:458] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: This master's current role is: LEADER
I20260618 13:59:27.891055 28797 tablet_bootstrap.cc:654] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.894275 29183 raft_consensus.cc:493] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.896237 29183 raft_consensus.cc:515] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.897388 28798 tablet_bootstrap.cc:492] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:27.897918 29183 leader_election.cc:290] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:27.898021 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fddea8fba4da40d7a5780e6750caf5e9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:27.898851 28798 ts_tablet_manager.cc:1403] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.007s
I20260618 13:59:27.898996 28799 tablet_bootstrap.cc:492] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.899194 28799 ts_tablet_manager.cc:1403] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.096s user 0.046s sys 0.044s
I20260618 13:59:27.898819 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fddea8fba4da40d7a5780e6750caf5e9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:27.899861 28346 leader_election.cc:343] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:27.900061 28346 leader_election.cc:343] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:27.899982 28798 raft_consensus.cc:359] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.900148 28798 raft_consensus.cc:385] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.899976 28799 raft_consensus.cc:359] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.900182 28346 leader_election.cc:304] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:27.900235 28798 raft_consensus.cc:740] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.900257 28799 raft_consensus.cc:385] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.900332 28799 raft_consensus.cc:740] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.900420 29183 raft_consensus.cc:2749] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:27.900554 28799 consensus_queue.cc:260] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.901250 28799 ts_tablet_manager.cc:1434] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:27.901786 28799 tablet_bootstrap.cc:492] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.902088 28798 consensus_queue.cc:260] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.902302 29208 raft_consensus.cc:493] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.902462 29208 raft_consensus.cc:515] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.902626 28798 ts_tablet_manager.cc:1434] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:27.903111 29208 leader_election.cc:290] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:27.903780 28617 leader_election.cc:343] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 8b59255b14df4158afe37cb8890e7415
I20260618 13:59:27.904126 28798 tablet_bootstrap.cc:492] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:27.904950 28797 tablet_bootstrap.cc:492] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.905083 28797 ts_tablet_manager.cc:1403] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.002s
I20260618 13:59:27.905855 28797 raft_consensus.cc:359] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.906044 28798 tablet_bootstrap.cc:654] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.906047 28797 raft_consensus.cc:385] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.906140 28797 raft_consensus.cc:740] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.906378 28797 consensus_queue.cc:260] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.912791 28797 ts_tablet_manager.cc:1434] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260618 13:59:27.913575 28799 tablet_bootstrap.cc:654] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.915066 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b59255b14df4158afe37cb8890e7415" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:27.915664 28618 leader_election.cc:343] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:27.915823 28618 leader_election.cc:304] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:27.916252 29208 raft_consensus.cc:2749] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:27.916831 28797 tablet_bootstrap.cc:492] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.921933 28799 tablet_bootstrap.cc:492] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.922137 28799 ts_tablet_manager.cc:1403] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.003s
I20260618 13:59:27.922952 28797 tablet_bootstrap.cc:654] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.925392 28799 raft_consensus.cc:359] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.925642 28799 raft_consensus.cc:385] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.925740 28799 raft_consensus.cc:740] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.926012 28799 consensus_queue.cc:260] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.926625 28799 ts_tablet_manager.cc:1434] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260618 13:59:27.927237 28797 tablet_bootstrap.cc:492] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.927271 28799 tablet_bootstrap.cc:492] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.927383 28797 ts_tablet_manager.cc:1403] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260618 13:59:27.928176 28797 raft_consensus.cc:359] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.928364 28797 raft_consensus.cc:385] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.928432 28797 raft_consensus.cc:740] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.928654 28797 consensus_queue.cc:260] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.929292 28797 ts_tablet_manager.cc:1434] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:27.929888 28797 tablet_bootstrap.cc:492] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.930997 28799 tablet_bootstrap.cc:654] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.932291 28797 tablet_bootstrap.cc:654] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.934892 29183 raft_consensus.cc:493] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.935102 29183 raft_consensus.cc:515] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.935935 29183 leader_election.cc:290] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:27.936590 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b88ed7cf79c4929a8704f66691cbed4" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:27.937386 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b88ed7cf79c4929a8704f66691cbed4" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:27.938128 29183 raft_consensus.cc:493] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.938320 29183 raft_consensus.cc:515] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:27.938125 28346 leader_election.cc:343] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:27.938696 28346 leader_election.cc:343] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:27.938824 28346 leader_election.cc:304] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:27.939143 29183 leader_election.cc:290] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:27.939339 29183 raft_consensus.cc:2749] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:27.942353 29183 raft_consensus.cc:493] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.942540 29183 raft_consensus.cc:515] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.943313 29183 leader_election.cc:290] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:27.947206 28798 tablet_bootstrap.cc:492] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:27.947384 28798 ts_tablet_manager.cc:1403] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.044s user 0.015s sys 0.007s
I20260618 13:59:27.947616 28799 tablet_bootstrap.cc:492] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.947767 28799 ts_tablet_manager.cc:1403] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.010s
I20260618 13:59:27.948040 28797 tablet_bootstrap.cc:492] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:27.948156 28797 ts_tablet_manager.cc:1403] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.002s
I20260618 13:59:27.948139 28798 raft_consensus.cc:359] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.948304 28798 raft_consensus.cc:385] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.948360 28798 raft_consensus.cc:740] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.948576 28798 consensus_queue.cc:260] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.948503 28799 raft_consensus.cc:359] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.948747 28799 raft_consensus.cc:385] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.948815 28799 raft_consensus.cc:740] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.948971 28798 ts_tablet_manager.cc:1434] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260618 13:59:27.948925 28797 raft_consensus.cc:359] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.949092 28797 raft_consensus.cc:385] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.949047 28799 consensus_queue.cc:260] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.949172 28797 raft_consensus.cc:740] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.949288 28798 tablet_bootstrap.cc:492] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:27.949399 28797 consensus_queue.cc:260] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.949548 28799 ts_tablet_manager.cc:1434] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:27.949828 28799 tablet_bootstrap.cc:492] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.950681 29230 raft_consensus.cc:493] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.951248 29230 raft_consensus.cc:515] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.951903 29230 leader_election.cc:290] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:27.952951 28799 tablet_bootstrap.cc:654] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:27.954559 28483 leader_election.cc:343] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: eead369de58f40378254697cbbbc6958
I20260618 13:59:27.954967 28797 ts_tablet_manager.cc:1434] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260618 13:59:27.955091 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eead369de58f40378254697cbbbc6958" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:27.955286 28682 raft_consensus.cc:2468] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:27.955742 28484 leader_election.cc:304] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:27.956094 29221 raft_consensus.cc:2804] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:27.956171 29221 raft_consensus.cc:493] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:27.956259 29221 raft_consensus.cc:3060] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:27.959139 28797 tablet_bootstrap.cc:492] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:27.961387 28797 tablet_bootstrap.cc:654] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.962731 29221 raft_consensus.cc:515] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.963963 28799 tablet_bootstrap.cc:492] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.964148 28799 ts_tablet_manager.cc:1403] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.008s
I20260618 13:59:27.964176 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eead369de58f40378254697cbbbc6958" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:27.964392 28682 raft_consensus.cc:3060] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:27.965102 29221 leader_election.cc:290] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:27.965104 28799 raft_consensus.cc:359] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:27.965314 28799 raft_consensus.cc:385] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.965425 28799 raft_consensus.cc:740] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
W20260618 13:59:27.965526 28483 leader_election.cc:343] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: eead369de58f40378254697cbbbc6958
I20260618 13:59:27.965679 28799 consensus_queue.cc:260] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:27.966347 28799 ts_tablet_manager.cc:1434] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:27.966411 28798 tablet_bootstrap.cc:654] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.966769 28799 tablet_bootstrap.cc:492] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.967085 28682 raft_consensus.cc:2468] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:27.967509 29208 raft_consensus.cc:493] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.967641 29208 raft_consensus.cc:515] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.967607 28484 leader_election.cc:304] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:27.968276 29208 leader_election.cc:290] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:27.968408 29221 raft_consensus.cc:2804] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:27.968549 29221 raft_consensus.cc:697] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:27.968837 28799 tablet_bootstrap.cc:654] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.968768 29221 consensus_queue.cc:237] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:27.969518 28617 leader_election.cc:343] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 125bb6885ab348d885d5a3b4bdd45bc7
I20260618 13:59:27.972030 29208 raft_consensus.cc:493] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.972221 29208 raft_consensus.cc:515] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.972975 29208 leader_election.cc:290] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:27.974737 28617 leader_election.cc:343] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 864f2602840c40fc8606b3784e671cba
I20260618 13:59:27.975693 29293 raft_consensus.cc:493] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.975867 29293 raft_consensus.cc:515] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.976092 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864f2602840c40fc8606b3784e671cba" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:27.976282 28549 raft_consensus.cc:2468] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:27.976663 29293 leader_election.cc:290] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:27.976832 28618 leader_election.cc:304] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:27.977624 29208 raft_consensus.cc:2804] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:27.977722 29208 raft_consensus.cc:493] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:27.977823 29208 raft_consensus.cc:3060] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:27.980402 29208 raft_consensus.cc:515] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.981343 28799 tablet_bootstrap.cc:492] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.981428 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125bb6885ab348d885d5a3b4bdd45bc7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:27.981539 28799 ts_tablet_manager.cc:1403] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.009s
I20260618 13:59:27.981885 29208 leader_election.cc:290] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:27.982159 29293 raft_consensus.cc:493] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.982178 28548 raft_consensus.cc:2468] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:27.982340 29293 raft_consensus.cc:515] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:27.982527 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864f2602840c40fc8606b3784e671cba" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:27.982585 28617 leader_election.cc:343] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 864f2602840c40fc8606b3784e671cba
I20260618 13:59:27.982519 28799 raft_consensus.cc:359] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:27.982678 28549 raft_consensus.cc:3060] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:27.982710 28799 raft_consensus.cc:385] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.982795 28799 raft_consensus.cc:740] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.982964 29293 leader_election.cc:290] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:27.983078 28799 consensus_queue.cc:260] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:27.983650 28799 ts_tablet_manager.cc:1434] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:27.983667 29208 raft_consensus.cc:493] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.983834 29208 raft_consensus.cc:515] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.983949 28799 tablet_bootstrap.cc:492] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:27.985209 28617 leader_election.cc:343] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 94365c44b5e24acab5df60c67203794a
I20260618 13:59:27.985355 29208 leader_election.cc:290] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:27.985628 28799 tablet_bootstrap.cc:654] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:27.986006 28618 leader_election.cc:304] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:27.986394 29208 raft_consensus.cc:2804] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:27.986603 29208 raft_consensus.cc:493] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:27.986687 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94365c44b5e24acab5df60c67203794a" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:27.986799 29208 raft_consensus.cc:3060] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:27.986964 28548 raft_consensus.cc:2468] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:27.987459 28618 leader_election.cc:304] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:27.987713 29186 raft_consensus.cc:2804] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:27.987813 29186 raft_consensus.cc:493] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:27.987890 29186 raft_consensus.cc:3060] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:27.988173 28799 tablet_bootstrap.cc:492] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:27.988262 28799 ts_tablet_manager.cc:1403] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260618 13:59:27.988732 28799 raft_consensus.cc:359] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:27.988837 28799 raft_consensus.cc:385] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:27.988891 28799 raft_consensus.cc:740] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:27.989032 28799 consensus_queue.cc:260] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:27.989356 28549 raft_consensus.cc:2468] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:27.989837 28799 ts_tablet_manager.cc:1434] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 13:59:27.989933 28618 leader_election.cc:304] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:27.990049 28799 tablet_bootstrap.cc:492] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:27.990558 29299 raft_consensus.cc:2804] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:27.991891 28799 tablet_bootstrap.cc:654] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:27.992259 28346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 53574.8us
I20260618 13:59:27.992887 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e86aeb21427e4859a5266f4451469b25" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:27.994864 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e86aeb21427e4859a5266f4451469b25" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:27.995997 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b75ece59d924e3db49e002bf0faa89c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:27.997341 29293 raft_consensus.cc:493] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:27.997532 29293 raft_consensus.cc:515] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:27.998090 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b75ece59d924e3db49e002bf0faa89c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:27.998319 29293 leader_election.cc:290] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:27.999050 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c557afc3abfa4ef188bf248e89f49d87" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:27.999797 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c557afc3abfa4ef188bf248e89f49d87" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:27.999841 28799 tablet_bootstrap.cc:492] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.000025 28799 ts_tablet_manager.cc:1403] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.006s
I20260618 13:59:28.000567 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8cf927bed9437086efd8e0f21ba26a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.000921 28799 raft_consensus.cc:359] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.001204 28799 raft_consensus.cc:385] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.001294 28799 raft_consensus.cc:740] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.001328 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8cf927bed9437086efd8e0f21ba26a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.001546 28799 consensus_queue.cc:260] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.002341 28799 ts_tablet_manager.cc:1434] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:28.002758 28799 tablet_bootstrap.cc:492] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.004683 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b16f7d3c7837498e996bba1818110642" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.005447 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b16f7d3c7837498e996bba1818110642" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.005501 28799 tablet_bootstrap.cc:654] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:28.006072 28346 leader_election.cc:343] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.007575 29303 raft_consensus.cc:493] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.007807 29303 raft_consensus.cc:515] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:28.008594 28346 leader_election.cc:343] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.008999 29299 raft_consensus.cc:697] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.009325 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b9c0754beb4e0e8ed3a7c1e9f368a9" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.009217 29303 leader_election.cc:290] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:28.009821 28618 leader_election.cc:343] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.010164 28617 leader_election.cc:343] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 75b9c0754beb4e0e8ed3a7c1e9f368a9
W20260618 13:59:28.010275 28346 leader_election.cc:343] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.010501 28346 leader_election.cc:343] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.010452 28617 leader_election.cc:304] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
W20260618 13:59:28.010673 28346 leader_election.cc:343] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.010861 28346 leader_election.cc:343] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.010962 28346 leader_election.cc:304] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.011260 29299 consensus_queue.cc:237] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.012687 29303 raft_consensus.cc:2749] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.013871 28799 tablet_bootstrap.cc:492] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.014035 28799 ts_tablet_manager.cc:1403] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.005s
I20260618 13:59:28.014524 29293 raft_consensus.cc:2749] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.014897 28797 tablet_bootstrap.cc:492] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.014926 28799 raft_consensus.cc:359] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.015060 28797 ts_tablet_manager.cc:1403] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.056s user 0.008s sys 0.008s
I20260618 13:59:28.015089 28799 raft_consensus.cc:385] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.015187 28799 raft_consensus.cc:740] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.015239 28798 tablet_bootstrap.cc:492] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:28.015369 28798 ts_tablet_manager.cc:1403] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.066s user 0.003s sys 0.000s
I20260618 13:59:28.015384 28799 consensus_queue.cc:260] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.015864 28799 ts_tablet_manager.cc:1434] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:28.015930 28797 raft_consensus.cc:359] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.016098 28797 raft_consensus.cc:385] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.016180 28797 raft_consensus.cc:740] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.016319 28799 tablet_bootstrap.cc:492] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.016279 28798 raft_consensus.cc:359] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.016450 28798 raft_consensus.cc:385] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.016397 28797 consensus_queue.cc:260] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.016528 28798 raft_consensus.cc:740] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.016749 28798 consensus_queue.cc:260] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.017285 28798 ts_tablet_manager.cc:1434] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:28.017326 28797 ts_tablet_manager.cc:1434] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:28.017366 29186 raft_consensus.cc:515] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.017565 28798 tablet_bootstrap.cc:492] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:28.017668 28797 tablet_bootstrap.cc:492] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.017982 29186 leader_election.cc:290] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.018307 28799 tablet_bootstrap.cc:654] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:28.018543 28346 leader_election.cc:343] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.018662 28346 leader_election.cc:304] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:28.022740 28617 leader_election.cc:343] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 94365c44b5e24acab5df60c67203794a
I20260618 13:59:28.023799 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94365c44b5e24acab5df60c67203794a" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:28.023948 28549 raft_consensus.cc:3060] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.024998 29208 raft_consensus.cc:515] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.026362 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125bb6885ab348d885d5a3b4bdd45bc7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:28.026604 28548 raft_consensus.cc:3060] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.027359 28549 raft_consensus.cc:2468] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:28.028007 28618 leader_election.cc:304] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:28.028278 29186 raft_consensus.cc:2804] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.028412 29186 raft_consensus.cc:697] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.028644 29186 consensus_queue.cc:237] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.030195 28548 raft_consensus.cc:2468] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:28.033416 28618 leader_election.cc:304] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:28.033587 29293 raft_consensus.cc:2749] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:28.034192 28346 leader_election.cc:343] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.035076 28346 leader_election.cc:304] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:28.035303 28617 leader_election.cc:343] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 125bb6885ab348d885d5a3b4bdd45bc7
I20260618 13:59:28.035607 28798 tablet_bootstrap.cc:654] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:28.036125 28213 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet 39b06ae7444a48dbbec9e096fd7e4052 (table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 39b06ae7444a48dbbec9e096fd7e4052 consensus state has no leader
W20260618 13:59:28.036202 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 3a0039a463ec4a9c82a21a3a69b565fc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3a0039a463ec4a9c82a21a3a69b565fc already in progress: creating tablet
W20260618 13:59:28.036286 28259 rpcz_store.cc:267] Call kudu.server.GenericService.CheckLeaks from 127.0.0.1:45110 (request call id 33) took 802 ms (client timeout 1000 ms). Trace:
W20260618 13:59:28.036417 28215 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.100.3:35503, user_credentials={real_user=slave}} blocked reactor thread for 778990us
I20260618 13:59:28.036779 28239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 488 } } ignore_live_leader: false dest_uuid: "7637285570574a6dbcef1de6f84ecac0" is_pre_election: true
I20260618 13:59:28.037127 28226 master_service.cc:960] ReplaceTablet: received request to replace tablet 283523e5b22843ba935eabe6283d3a99 from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:28.037587 28071 leader_election.cc:304] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e4d4f4566be42dfbafb7c0e40fb17ff; no voters: 7637285570574a6dbcef1de6f84ecac0, a950ab7cd66b4cb0b29592aa692c2dd0
W20260618 13:59:28.037707 28221 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_473a0c23203b404ea8e5ee0aed201e7b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:28.037971 28222 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_152a483501b54a67bbdc0cae4959d452 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:28.036419 28259 rpcz_store.cc:269] 0618 13:59:27.233458 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:27.233511 (+ 53us) service_pool.cc:224] Handling call
0618 13:59:28.036238 (+802727us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260618 13:59:28.038120 28239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 500 } } ignore_live_leader: false dest_uuid: "7637285570574a6dbcef1de6f84ecac0" is_pre_election: true
I20260618 13:59:28.038220 29266 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:28.038544 29293 raft_consensus.cc:2749] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.039309 28238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 500 } } ignore_live_leader: false dest_uuid: "7637285570574a6dbcef1de6f84ecac0"
W20260618 13:59:28.040342 28346 leader_election.cc:343] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.040457 28346 leader_election.cc:304] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.040681 29293 raft_consensus.cc:2749] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:28.041043 28346 leader_election.cc:343] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.041139 28346 leader_election.cc:304] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:28.041508 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 3a0039a463ec4a9c82a21a3a69b565fc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3a0039a463ec4a9c82a21a3a69b565fc already in progress: creating tablet
W20260618 13:59:28.041719 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 75824a5d2f7e47838b60e6a2b785c6b2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75824a5d2f7e47838b60e6a2b785c6b2 already in progress: creating tablet
I20260618 13:59:28.041862 29293 raft_consensus.cc:2749] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.035918 29208 leader_election.cc:290] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:28.042353 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet c3d523419ff244f085d6eac1a3231973 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c3d523419ff244f085d6eac1a3231973 already in progress: creating tablet
W20260618 13:59:28.042889 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 7aa9f65d8a5c46fdb745ee17af946dce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7aa9f65d8a5c46fdb745ee17af946dce already in progress: creating tablet
I20260618 13:59:28.043020 28226 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_9a78338c3e414b0abea1281082202f57" } modify_external_catalogs: true
I20260618 13:59:28.043049 28225 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9c17e76ae9b44207998a43e3576908c4 (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]). Expected version 2 got 0
W20260618 13:59:28.043129 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet c7f42997416841a7b2c1ad347d2e9282 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c7f42997416841a7b2c1ad347d2e9282 already in progress: creating tablet
I20260618 13:59:28.043183 28226 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_9a78338c3e414b0abea1281082202f57" } modify_external_catalogs: true
W20260618 13:59:28.043365 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 41107eb1d9164a9c98060003b4027412 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 41107eb1d9164a9c98060003b4027412 already in progress: creating tablet
W20260618 13:59:28.043592 28215 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 7572d17724d249739be1a1cb9468f909 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7572d17724d249739be1a1cb9468f909 already in progress: creating tablet
W20260618 13:59:28.043671 28225 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.1:59103 (request call id 96) took 869 ms (client timeout 1000 ms). Trace:
W20260618 13:59:28.043773 28225 rpcz_store.cc:269] 0618 13:59:27.174265 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:27.198655 (+ 24390us) service_pool.cc:224] Handling call
0618 13:59:28.042554 (+843899us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x616000041b54. stack: 00007f11a1c2f496 00007f11a1c2f250 00007f11a0c5a3ad 000000000099c914 000000000099c8dd 00007f11b11106bd 00007f11b11107b3 00007f11adcd8b21 00007f11b5a3d95d 00007f11b5bcaf08 00007f11ac629ca6 00007f11ac6299d5 00007f11a8d3988e 00007f11a8d3876e 00007f11a8d3e9a8 00007f11a8d3f30f
0618 13:59:28.043608 (+ 1054us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":29341056}
I20260618 13:59:28.044494 28225 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_3ef2ff183a6044e2a9e069a4eac133b8" } modify_external_catalogs: true
I20260618 13:59:28.044696 28225 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_3ef2ff183a6044e2a9e069a4eac133b8" } modify_external_catalogs: true
I20260618 13:59:28.050594 28223 master_service.cc:960] ReplaceTablet: received request to replace tablet ca2ee746c5024cfc9b31c470f9616e7d from {username='slave'} at 127.0.0.1:45138
I20260618 13:59:28.056582 28799 tablet_bootstrap.cc:492] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.056756 28799 ts_tablet_manager.cc:1403] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.040s user 0.007s sys 0.015s
I20260618 13:59:28.058768 28227 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45138:
name: "default.Table_2dc7e2d92c504d7f91ac02de7c2bdee4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:28.059259 28799 raft_consensus.cc:359] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.059392 28799 raft_consensus.cc:385] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.059438 28799 raft_consensus.cc:740] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
W20260618 13:59:28.059446 28227 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2dc7e2d92c504d7f91ac02de7c2bdee4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:28.059688 28799 consensus_queue.cc:260] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.060130 28799 ts_tablet_manager.cc:1434] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260618 13:59:28.060587 28799 tablet_bootstrap.cc:492] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.063189 28799 tablet_bootstrap.cc:654] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:28.063717 28228 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.3:55949 (request call id 87) took 817 ms (client timeout 1000 ms). Trace:
W20260618 13:59:28.063777 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 41107eb1d9164a9c98060003b4027412 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 41107eb1d9164a9c98060003b4027412 already in progress: creating tablet
W20260618 13:59:28.063827 28228 rpcz_store.cc:269] 0618 13:59:28.036750 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:28.038751 (+ 2001us) service_pool.cc:224] Handling call
0618 13:59:28.063656 (+ 24905us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":2}
W20260618 13:59:28.064038 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet c7f42997416841a7b2c1ad347d2e9282 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c7f42997416841a7b2c1ad347d2e9282 already in progress: creating tablet
W20260618 13:59:28.064239 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 75824a5d2f7e47838b60e6a2b785c6b2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75824a5d2f7e47838b60e6a2b785c6b2 already in progress: creating tablet
I20260618 13:59:28.064201 28228 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_fd6ef45aeec84b48b8fb7a54db4b4f81" } modify_external_catalogs: true
I20260618 13:59:28.064383 28228 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45138:
table { table_name: "default.Table_fd6ef45aeec84b48b8fb7a54db4b4f81" } modify_external_catalogs: true
I20260618 13:59:28.064491 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 68b8293a7c924a449307ab88a1cf9290 (table default.Table_c97dcabd4c2b45a18f445a099ec7ace9 [id=38bde7a301814aa285e4bd30bcc527aa]) successfully deleted
I20260618 13:59:28.065130 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 8be4374ae7cf4907940ae90299bf9ba0 (table default.Table_0a13b90182654346acbbc8f47a56436d [id=c31d65adb55f458c9ced14d598464f89]) successfully deleted
I20260618 13:59:28.065352 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet e025b64380314546af8a1ab90c817825 (table default.Table_26e7d8c4f91144268a2a43e83992667d [id=803d5220ce1a4dcfaf27d93c574be2ac]) successfully deleted
I20260618 13:59:28.065480 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet fca76774f119435899188313a33dad98 (table default.Table_4cf88aaddada4361837d7d49170f029b [id=50525d6257fa4de8a00dbee44d37f24d]) successfully deleted
W20260618 13:59:28.065730 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 7aa9f65d8a5c46fdb745ee17af946dce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7aa9f65d8a5c46fdb745ee17af946dce already in progress: creating tablet
W20260618 13:59:28.065929 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 7572d17724d249739be1a1cb9468f909 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7572d17724d249739be1a1cb9468f909 already in progress: creating tablet
W20260618 13:59:28.066133 28215 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet c3d523419ff244f085d6eac1a3231973 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c3d523419ff244f085d6eac1a3231973 already in progress: creating tablet
I20260618 13:59:28.066304 28215 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 0a51e830e3014726b1d1c2608973f036 (table default.Table_696ead97d50747228517c265538b25fb [id=716ae493050c4288b9bf041c94523b05]) successfully deleted
I20260618 13:59:28.068029 28048 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu with pid 28195
I20260618 13:59:28.068169 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet 08a2dee3ac5a4bec96445a6001717eda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:34092
I20260618 13:59:28.045363 28797 tablet_bootstrap.cc:654] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.069921 28097 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 2 FOLLOWER]: Rejecting Update request from peer 7637285570574a6dbcef1de6f84ecac0 for earlier term 1. Current term is 2. Ops: [1.489-1.509]
I20260618 13:59:28.067966 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 125bb6885ab348d885d5a3b4bdd45bc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:28.068219 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet 94365c44b5e24acab5df60c67203794a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43976
I20260618 13:59:28.036026 29301 raft_consensus.cc:2804] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.079030 29301 raft_consensus.cc:697] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.079489 29311 tablet_replica.cc:333] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:28.079855 29311 raft_consensus.cc:2243] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:28.079691 29301 consensus_queue.cc:237] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.080158 29311 raft_consensus.cc:2272] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:28.081148 28797 tablet_bootstrap.cc:492] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.081534 28797 ts_tablet_manager.cc:1403] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.064s user 0.007s sys 0.002s
I20260618 13:59:28.081962 29311 ts_tablet_manager.cc:1916] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:28.082698 28797 raft_consensus.cc:359] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.082921 28797 raft_consensus.cc:385] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.083035 28797 raft_consensus.cc:740] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.083273 28797 consensus_queue.cc:260] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.083812 28797 ts_tablet_manager.cc:1434] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:28.084316 28797 tablet_bootstrap.cc:492] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.086550 28797 tablet_bootstrap.cc:654] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.089865 29293 raft_consensus.cc:493] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.090031 29293 raft_consensus.cc:515] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.090143 29221 raft_consensus.cc:493] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.090266 29221 raft_consensus.cc:515] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.090711 29293 leader_election.cc:290] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.091079 29221 leader_election.cc:290] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:28.091382 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fae928833e44d2ba860d6184d4c0142" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.092051 29311 ts_tablet_manager.cc:1929] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:28.092554 28797 tablet_bootstrap.cc:492] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.092700 28797 ts_tablet_manager.cc:1403] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
W20260618 13:59:28.092772 28346 leader_election.cc:343] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.093495 28797 raft_consensus.cc:359] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.093705 28797 raft_consensus.cc:385] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.093783 28797 raft_consensus.cc:740] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.094038 28797 consensus_queue.cc:260] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.094571 28797 ts_tablet_manager.cc:1434] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:28.094856 28797 tablet_bootstrap.cc:492] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.096554 28797 tablet_bootstrap.cc:654] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.106525 29313 tablet_replica.cc:333] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:28.106841 28799 tablet_bootstrap.cc:492] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.106946 29313 raft_consensus.cc:2243] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:28.107000 28799 ts_tablet_manager.cc:1403] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.046s user 0.020s sys 0.003s
I20260618 13:59:28.107297 29313 pending_rounds.cc:70] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257: Trying to abort 1 pending ops.
I20260618 13:59:28.107384 29313 pending_rounds.cc:77] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298216624458731520 op_type: NO_OP noop_request { }
I20260618 13:59:28.107540 29313 raft_consensus.cc:2889] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:28.107604 29311 log.cc:1199] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/08a2dee3ac5a4bec96445a6001717eda
I20260618 13:59:28.107641 29313 raft_consensus.cc:2272] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:28.107889 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fae928833e44d2ba860d6184d4c0142" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.108119 29311 ts_tablet_manager.cc:1950] T 08a2dee3ac5a4bec96445a6001717eda P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:28.108059 28799 raft_consensus.cc:359] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.108270 28799 raft_consensus.cc:385] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.108366 28799 raft_consensus.cc:740] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
W20260618 13:59:28.108520 28483 leader_election.cc:343] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: e025b64380314546af8a1ab90c817825
I20260618 13:59:28.108608 28799 consensus_queue.cc:260] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:28.108889 28346 leader_election.cc:343] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.109040 28346 leader_election.cc:304] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.109258 29293 raft_consensus.cc:2749] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.109551 28797 tablet_bootstrap.cc:492] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.109700 28797 ts_tablet_manager.cc:1403] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.001s
I20260618 13:59:28.109690 28799 ts_tablet_manager.cc:1434] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:28.109965 28799 tablet_bootstrap.cc:492] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.110381 28797 raft_consensus.cc:359] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.110606 28797 raft_consensus.cc:385] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.110702 28797 raft_consensus.cc:740] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.111301 29313 ts_tablet_manager.cc:1916] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:28.111768 28799 tablet_bootstrap.cc:654] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:28.113003 28484 leader_election.cc:343] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: e025b64380314546af8a1ab90c817825
I20260618 13:59:28.113978 28484 leader_election.cc:304] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.114460 29221 raft_consensus.cc:2749] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.115881 28798 tablet_bootstrap.cc:492] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:28.116058 28798 ts_tablet_manager.cc:1403] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.099s user 0.005s sys 0.011s
I20260618 13:59:28.116338 28797 consensus_queue.cc:260] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.116760 28798 raft_consensus.cc:359] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.116952 28798 raft_consensus.cc:385] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.117017 28798 raft_consensus.cc:740] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.117342 28798 consensus_queue.cc:260] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.117934 28798 ts_tablet_manager.cc:1434] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:28.119488 28799 tablet_bootstrap.cc:492] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.119638 28799 ts_tablet_manager.cc:1403] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.006s
I20260618 13:59:28.120003 28798 tablet_bootstrap.cc:492] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:28.120361 28799 raft_consensus.cc:359] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.120551 28799 raft_consensus.cc:385] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.120620 28799 raft_consensus.cc:740] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.120842 28799 consensus_queue.cc:260] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.121404 28799 ts_tablet_manager.cc:1434] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:28.121685 28799 tablet_bootstrap.cc:492] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.122324 29301 raft_consensus.cc:493] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.122490 29301 raft_consensus.cc:515] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.123081 29301 leader_election.cc:290] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.123279 28799 tablet_bootstrap.cc:654] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.123317 28797 ts_tablet_manager.cc:1434] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
W20260618 13:59:28.124300 28617 leader_election.cc:343] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: c552d1316e3340f4bcfa39618ae309b8
I20260618 13:59:28.125821 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c552d1316e3340f4bcfa39618ae309b8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.126627 28798 tablet_bootstrap.cc:654] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:28.126621 28618 leader_election.cc:343] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.126734 28618 leader_election.cc:304] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.126822 28797 tablet_bootstrap.cc:492] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.127571 29301 raft_consensus.cc:2749] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.136922 28797 tablet_bootstrap.cc:654] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.138924 29313 ts_tablet_manager.cc:1929] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:28.139057 29313 log.cc:1199] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/125bb6885ab348d885d5a3b4bdd45bc7
I20260618 13:59:28.139484 29313 ts_tablet_manager.cc:1950] T 125bb6885ab348d885d5a3b4bdd45bc7 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:28.139540 28799 tablet_bootstrap.cc:492] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.139673 28799 ts_tablet_manager.cc:1403] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.018s user 0.017s sys 0.000s
I20260618 13:59:28.140228 28797 tablet_bootstrap.cc:492] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.140492 28797 ts_tablet_manager.cc:1403] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.002s
I20260618 13:59:28.140494 28799 raft_consensus.cc:359] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.140671 28799 raft_consensus.cc:385] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.140774 28799 raft_consensus.cc:740] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.141050 28799 consensus_queue.cc:260] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.141498 29313 tablet_replica.cc:333] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:28.141686 28799 ts_tablet_manager.cc:1434] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:28.141805 29313 raft_consensus.cc:2243] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:28.142063 29313 pending_rounds.cc:70] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257: Trying to abort 1 pending ops.
I20260618 13:59:28.142143 29313 pending_rounds.cc:77] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298216624247496704 op_type: NO_OP noop_request { }
I20260618 13:59:28.142308 29313 raft_consensus.cc:2889] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:28.142472 29313 raft_consensus.cc:2272] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:28.142508 28799 tablet_bootstrap.cc:492] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.143262 29313 ts_tablet_manager.cc:1916] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:28.143875 28798 tablet_bootstrap.cc:492] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:28.144034 28798 ts_tablet_manager.cc:1403] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20260618 13:59:28.144510 28799 tablet_bootstrap.cc:654] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.144773 28798 raft_consensus.cc:359] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.144970 28798 raft_consensus.cc:385] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.145080 28798 raft_consensus.cc:740] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.145358 28798 consensus_queue.cc:260] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.145949 28798 ts_tablet_manager.cc:1434] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:28.146209 28798 tablet_bootstrap.cc:492] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:28.150944 28798 tablet_bootstrap.cc:654] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.152554 28799 tablet_bootstrap.cc:492] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.152740 28799 ts_tablet_manager.cc:1403] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.002s
I20260618 13:59:28.153621 28799 raft_consensus.cc:359] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.153862 28799 raft_consensus.cc:385] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.154047 28799 raft_consensus.cc:740] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.154369 28799 consensus_queue.cc:260] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.153083 28797 raft_consensus.cc:359] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.154693 28797 raft_consensus.cc:385] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.155309 28799 ts_tablet_manager.cc:1434] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260618 13:59:28.155683 28799 tablet_bootstrap.cc:492] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.157665 29313 ts_tablet_manager.cc:1929] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:28.157888 29313 log.cc:1199] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/94365c44b5e24acab5df60c67203794a
I20260618 13:59:28.158185 28799 tablet_bootstrap.cc:654] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.158501 29313 ts_tablet_manager.cc:1950] T 94365c44b5e24acab5df60c67203794a P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:28.158804 29221 raft_consensus.cc:493] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.158950 29221 raft_consensus.cc:515] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.159614 29221 leader_election.cc:290] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.160315 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da4d4f03911f4ac099ecc2fa1d8af494" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.160486 29301 raft_consensus.cc:493] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.160526 28681 raft_consensus.cc:2468] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:28.160637 29301 raft_consensus.cc:515] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:28.160717 28483 leader_election.cc:343] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: da4d4f03911f4ac099ecc2fa1d8af494
I20260618 13:59:28.161038 28484 leader_election.cc:304] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:28.161247 29301 leader_election.cc:290] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.161311 28799 tablet_bootstrap.cc:492] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.161440 28799 ts_tablet_manager.cc:1403] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260618 13:59:28.161494 29221 raft_consensus.cc:2804] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.161600 29221 raft_consensus.cc:493] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.161682 29221 raft_consensus.cc:3060] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.154845 28797 raft_consensus.cc:740] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.162039 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f81428f804a485c967026c4527fd250" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:28.162245 28411 raft_consensus.cc:2468] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:28.162364 28799 raft_consensus.cc:359] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.162559 28799 raft_consensus.cc:385] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.162638 28617 leader_election.cc:304] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:28.162669 28799 raft_consensus.cc:740] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.162892 28799 consensus_queue.cc:260] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.163260 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f81428f804a485c967026c4527fd250" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.163527 28799 ts_tablet_manager.cc:1434] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260618 13:59:28.163741 28618 leader_election.cc:343] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.163983 28799 tablet_bootstrap.cc:492] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.164044 29301 raft_consensus.cc:2804] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.164129 29301 raft_consensus.cc:493] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.164229 29301 raft_consensus.cc:3060] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.164337 29221 raft_consensus.cc:515] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.165545 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da4d4f03911f4ac099ecc2fa1d8af494" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:28.165735 28681 raft_consensus.cc:3060] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.165781 28799 tablet_bootstrap.cc:654] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.167084 29301 raft_consensus.cc:515] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.168443 28681 raft_consensus.cc:2468] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:28.169217 29221 leader_election.cc:290] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.169456 29186 raft_consensus.cc:493] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.169601 29186 raft_consensus.cc:515] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.170184 29186 leader_election.cc:290] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:28.171118 28617 leader_election.cc:343] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: ac58e939730c4af681c2f08bcfbb6602
I20260618 13:59:28.168479 28797 consensus_queue.cc:260] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.171875 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f81428f804a485c967026c4527fd250" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:28.172168 28797 ts_tablet_manager.cc:1434] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.032s user 0.001s sys 0.001s
I20260618 13:59:28.172627 28797 tablet_bootstrap.cc:492] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:28.172789 28483 leader_election.cc:343] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: da4d4f03911f4ac099ecc2fa1d8af494
I20260618 13:59:28.173763 28798 tablet_bootstrap.cc:492] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:28.173911 28798 ts_tablet_manager.cc:1403] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.001s
I20260618 13:59:28.174737 28798 raft_consensus.cc:359] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.174906 28798 raft_consensus.cc:385] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.174978 28798 raft_consensus.cc:740] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.175236 28798 consensus_queue.cc:260] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.176968 28798 ts_tablet_manager.cc:1434] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260618 13:59:28.178117 28618 leader_election.cc:343] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.178329 28797 tablet_bootstrap.cc:654] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.177861 28484 leader_election.cc:304] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:28.180498 29221 raft_consensus.cc:2804] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.180629 29221 raft_consensus.cc:697] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:28.180868 29221 consensus_queue.cc:237] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.182672 29186 raft_consensus.cc:493] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.182824 29186 raft_consensus.cc:515] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.183347 29186 leader_election.cc:290] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:28.184172 28617 leader_election.cc:343] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 36b8a46a41e543c58a946b6764618c7d
I20260618 13:59:28.186825 28797 tablet_bootstrap.cc:492] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.186944 28797 ts_tablet_manager.cc:1403] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.001s
I20260618 13:59:28.187282 29301 leader_election.cc:290] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.187881 28797 raft_consensus.cc:359] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.188045 28797 raft_consensus.cc:385] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.188089 29293 raft_consensus.cc:493] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.188140 28797 raft_consensus.cc:740] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.188225 29293 raft_consensus.cc:515] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.188354 28797 consensus_queue.cc:260] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.188814 29293 leader_election.cc:290] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.188862 28797 ts_tablet_manager.cc:1434] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:28.189218 28797 tablet_bootstrap.cc:492] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.189675 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283523e5b22843ba935eabe6283d3a99" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:28.190157 28346 leader_election.cc:343] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.190495 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283523e5b22843ba935eabe6283d3a99" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.177989 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac58e939730c4af681c2f08bcfbb6602" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.191116 28797 tablet_bootstrap.cc:654] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:28.191316 28346 leader_election.cc:343] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.191432 28346 leader_election.cc:304] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.191854 29293 raft_consensus.cc:2749] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.192623 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f81428f804a485c967026c4527fd250" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:28.192884 28411 raft_consensus.cc:3060] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.193701 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36b8a46a41e543c58a946b6764618c7d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.193866 28549 raft_consensus.cc:2468] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:28.194300 28618 leader_election.cc:343] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.194629 28618 leader_election.cc:304] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.195154 28618 leader_election.cc:304] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:28.195408 29186 raft_consensus.cc:2749] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.195633 29186 raft_consensus.cc:2804] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.195737 29186 raft_consensus.cc:493] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.195888 29186 raft_consensus.cc:3060] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.196543 28411 raft_consensus.cc:2468] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:28.199084 28617 leader_election.cc:304] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.199421 29299 raft_consensus.cc:2804] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.199612 29299 raft_consensus.cc:697] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.199877 29299 consensus_queue.cc:237] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.177273 28798 tablet_bootstrap.cc:492] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:28.198556 29186 raft_consensus.cc:515] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.202641 28798 tablet_bootstrap.cc:654] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:28.203718 28617 leader_election.cc:343] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 36b8a46a41e543c58a946b6764618c7d
I20260618 13:59:28.204331 29186 leader_election.cc:290] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.204560 28799 tablet_bootstrap.cc:492] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.204759 28799 ts_tablet_manager.cc:1403] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.041s user 0.006s sys 0.016s
I20260618 13:59:28.204908 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36b8a46a41e543c58a946b6764618c7d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:28.205083 28549 raft_consensus.cc:3060] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.205879 28799 raft_consensus.cc:359] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.206314 28799 raft_consensus.cc:385] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.206444 28799 raft_consensus.cc:740] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.208247 28549 raft_consensus.cc:2468] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:28.208770 28618 leader_election.cc:304] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:28.209360 29186 raft_consensus.cc:2804] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.209568 29186 raft_consensus.cc:697] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.210160 29186 consensus_queue.cc:237] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.214687 28799 consensus_queue.cc:260] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.215417 28799 ts_tablet_manager.cc:1434] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260618 13:59:28.216120 28799 tablet_bootstrap.cc:492] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.217720 28797 tablet_bootstrap.cc:492] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.217864 28799 tablet_bootstrap.cc:654] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.217883 28797 ts_tablet_manager.cc:1403] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.008s
I20260618 13:59:28.218664 28797 raft_consensus.cc:359] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.218847 28797 raft_consensus.cc:385] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.218902 28797 raft_consensus.cc:740] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.219146 28797 consensus_queue.cc:260] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.221429 28797 ts_tablet_manager.cc:1434] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:28.221701 28797 tablet_bootstrap.cc:492] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.223954 28797 tablet_bootstrap.cc:654] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.229020 29299 raft_consensus.cc:493] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.229069 29293 raft_consensus.cc:493] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.229176 29299 raft_consensus.cc:515] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.229188 29293 raft_consensus.cc:515] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.229774 29293 leader_election.cc:290] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.229797 29299 leader_election.cc:290] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.230362 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316f470ea82f42fca48e4e96d49c9574" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:28.230947 28617 leader_election.cc:343] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: d6871232860b41d489e34d95eb897241
I20260618 13:59:28.231133 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6871232860b41d489e34d95eb897241" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.231377 28548 raft_consensus.cc:2468] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:28.231822 29293 raft_consensus.cc:493] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.231976 29293 raft_consensus.cc:515] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:28.232193 28346 leader_election.cc:343] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.232587 29293 leader_election.cc:290] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.233352 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6b01b897134d5bbd6a7d93b82ddb5e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:28.233748 28346 leader_election.cc:343] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.234086 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6b01b897134d5bbd6a7d93b82ddb5e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.234911 28799 tablet_bootstrap.cc:492] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
W20260618 13:59:28.234943 28346 leader_election.cc:343] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.235056 28346 leader_election.cc:304] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.235061 28799 ts_tablet_manager.cc:1403] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.006s
I20260618 13:59:28.235551 28618 leader_election.cc:304] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:28.235900 29299 raft_consensus.cc:2804] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.235873 28799 raft_consensus.cc:359] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.235996 29299 raft_consensus.cc:493] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.236029 28799 raft_consensus.cc:385] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.236068 29299 raft_consensus.cc:3060] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.236111 28799 raft_consensus.cc:740] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.236328 28799 consensus_queue.cc:260] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.236791 29183 raft_consensus.cc:2749] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.236930 28799 ts_tablet_manager.cc:1434] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:28.237182 28799 tablet_bootstrap.cc:492] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.231302 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316f470ea82f42fca48e4e96d49c9574" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.238952 28799 tablet_bootstrap.cc:654] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:28.239068 28346 leader_election.cc:343] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.239180 28346 leader_election.cc:304] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.239629 29293 raft_consensus.cc:2749] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.240561 29299 raft_consensus.cc:515] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.241223 29299 leader_election.cc:290] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.242539 28798 tablet_bootstrap.cc:492] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:28.242689 28798 ts_tablet_manager.cc:1403] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.066s user 0.004s sys 0.001s
I20260618 13:59:28.242681 29293 raft_consensus.cc:493] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.242832 29293 raft_consensus.cc:515] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.243332 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6871232860b41d489e34d95eb897241" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:28.243367 29293 leader_election.cc:290] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:28.243502 28549 raft_consensus.cc:3060] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.243449 28798 raft_consensus.cc:359] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.243690 28798 raft_consensus.cc:385] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.243798 28798 raft_consensus.cc:740] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
W20260618 13:59:28.243638 28617 leader_election.cc:343] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: d6871232860b41d489e34d95eb897241
I20260618 13:59:28.244481 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98b42d23832545ae81a7cdff19e4689f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.244508 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98b42d23832545ae81a7cdff19e4689f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.245002 28798 consensus_queue.cc:260] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:28.245168 28346 leader_election.cc:343] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.245355 28346 leader_election.cc:343] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.245471 28346 leader_election.cc:304] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.245707 28798 ts_tablet_manager.cc:1434] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260618 13:59:28.245939 29293 raft_consensus.cc:2749] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.246341 28798 tablet_bootstrap.cc:492] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:28.246776 29293 raft_consensus.cc:493] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.246939 29293 raft_consensus.cc:515] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.247505 29293 leader_election.cc:290] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.248617 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d667ea5f029f46d89fcb3be03069b97b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.251288 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d667ea5f029f46d89fcb3be03069b97b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.251467 28799 tablet_bootstrap.cc:492] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.251608 28799 ts_tablet_manager.cc:1403] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20260618 13:59:28.252344 28799 raft_consensus.cc:359] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.252537 28799 raft_consensus.cc:385] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.252611 28799 raft_consensus.cc:740] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.252884 28799 consensus_queue.cc:260] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.253477 28799 ts_tablet_manager.cc:1434] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:28.253772 28799 tablet_bootstrap.cc:492] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:28.253746 28346 leader_election.cc:343] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.253964 28346 leader_election.cc:343] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.254086 28798 tablet_bootstrap.cc:654] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.254079 28346 leader_election.cc:304] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.254503 29293 raft_consensus.cc:2749] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.255652 28799 tablet_bootstrap.cc:654] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.254194 28549 raft_consensus.cc:2468] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:28.259100 28618 leader_election.cc:304] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:28.259609 29186 raft_consensus.cc:2804] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.260123 29186 raft_consensus.cc:697] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.260773 29186 consensus_queue.cc:237] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.262814 28797 tablet_bootstrap.cc:492] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.262990 28797 ts_tablet_manager.cc:1403] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.041s user 0.001s sys 0.007s
I20260618 13:59:28.263973 28797 raft_consensus.cc:359] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.264220 28797 raft_consensus.cc:385] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.264340 28797 raft_consensus.cc:740] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.264669 28797 consensus_queue.cc:260] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.265492 28797 ts_tablet_manager.cc:1434] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:28.265805 28797 tablet_bootstrap.cc:492] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.267331 29293 raft_consensus.cc:493] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.267530 29293 raft_consensus.cc:515] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.268344 29293 leader_election.cc:290] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.271255 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6dd422c05324c989e29b20ae2afabad" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.271639 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6dd422c05324c989e29b20ae2afabad" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:28.274899 28346 leader_election.cc:343] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.275157 28346 leader_election.cc:343] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.275276 28346 leader_election.cc:304] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.275661 29293 raft_consensus.cc:2749] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.275717 28797 tablet_bootstrap.cc:654] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.280578 28799 tablet_bootstrap.cc:492] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.280737 28799 ts_tablet_manager.cc:1403] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.006s
I20260618 13:59:28.281275 29293 raft_consensus.cc:493] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.281424 29293 raft_consensus.cc:515] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.281978 29293 leader_election.cc:290] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.282706 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7a97176f72349cda221baaea61d18ad" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.282913 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7a97176f72349cda221baaea61d18ad" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.283517 28799 raft_consensus.cc:359] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.283712 28799 raft_consensus.cc:385] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.283803 28799 raft_consensus.cc:740] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.284008 28799 consensus_queue.cc:260] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.284515 28799 ts_tablet_manager.cc:1434] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260618 13:59:28.284837 28799 tablet_bootstrap.cc:492] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.284860 29183 raft_consensus.cc:493] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260618 13:59:28.285346 28346 leader_election.cc:343] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.285418 29183 raft_consensus.cc:515] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:28.285617 28346 leader_election.cc:343] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.285754 28346 leader_election.cc:304] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.286118 29183 leader_election.cc:290] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.286285 29183 raft_consensus.cc:2749] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.286595 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f3d2f236fd44ab8af0524ea282cfa94" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.286767 28799 tablet_bootstrap.cc:654] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.286731 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f3d2f236fd44ab8af0524ea282cfa94" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:28.287137 28346 leader_election.cc:343] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.287397 28346 leader_election.cc:343] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.287523 28346 leader_election.cc:304] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.287889 29293 raft_consensus.cc:2749] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:28.297747 28483 connection.cc:573] server connection from 127.0.0.1:34092 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260618 13:59:28.297757 28616 connection.cc:573] server connection from 127.0.0.1:43976 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260618 13:59:28.298084 29186 raft_consensus.cc:493] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.298324 29186 raft_consensus.cc:515] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.300710 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c00003300246ada2a356e3e1cf684b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:28.300782 28617 leader_election.cc:343] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: d9c00003300246ada2a356e3e1cf684b
I20260618 13:59:28.301003 28549 raft_consensus.cc:2468] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:28.301328 28983 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.100.60:42083 (error 108)
I20260618 13:59:28.301558 28618 leader_election.cc:304] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:28.301959 29299 raft_consensus.cc:2804] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.302068 29299 raft_consensus.cc:493] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.302209 29299 raft_consensus.cc:3060] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.305894 29299 raft_consensus.cc:515] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.307826 29299 leader_election.cc:290] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.308044 29186 leader_election.cc:290] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:28.308063 28736 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.27.100.60:42083)
W20260618 13:59:28.308713 28736 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.27.100.60:42083)
I20260618 13:59:28.308964 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c00003300246ada2a356e3e1cf684b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:28.309082 28736 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.27.100.60:42083)
I20260618 13:59:28.309214 28549 raft_consensus.cc:3060] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:28.309427 28736 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.27.100.60:42083)
I20260618 13:59:28.312671 28549 raft_consensus.cc:2468] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:28.313055 29186 raft_consensus.cc:493] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.313267 28618 leader_election.cc:304] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:28.313256 29186 raft_consensus.cc:515] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.313611 29299 raft_consensus.cc:2804] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.313777 29299 raft_consensus.cc:697] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.314095 29186 leader_election.cc:290] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.314574 29301 raft_consensus.cc:493] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.314499 29299 consensus_queue.cc:237] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.314713 29301 raft_consensus.cc:515] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.315258 29301 leader_election.cc:290] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.316646 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc72a06ee8e4420b07a17d2feb3fcc2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.316874 28549 raft_consensus.cc:2468] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:28.317108 28617 leader_election.cc:343] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: edc72a06ee8e4420b07a17d2feb3fcc2
I20260618 13:59:28.317376 28798 tablet_bootstrap.cc:492] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
W20260618 13:59:28.317353 28617 leader_election.cc:343] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: d9c00003300246ada2a356e3e1cf684b
I20260618 13:59:28.317525 28798 ts_tablet_manager.cc:1403] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.071s user 0.006s sys 0.002s
W20260618 13:59:28.317655 28617 leader_election.cc:343] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 37a6764a5e024903848561c9935b1a43
I20260618 13:59:28.318188 28618 leader_election.cc:304] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:28.318349 28798 raft_consensus.cc:359] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.318557 28798 raft_consensus.cc:385] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.318645 28798 raft_consensus.cc:740] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.318838 29301 raft_consensus.cc:2804] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.318936 29301 raft_consensus.cc:493] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.319074 29301 raft_consensus.cc:3060] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.319000 28798 consensus_queue.cc:260] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.319881 28798 ts_tablet_manager.cc:1434] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:28.320400 28798 tablet_bootstrap.cc:492] T 1f81428f804a485c967026c4527fd250 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:28.321776 29301 raft_consensus.cc:515] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.322355 28798 tablet_bootstrap.cc:654] T 1f81428f804a485c967026c4527fd250 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:28.323975 28617 leader_election.cc:343] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: edc72a06ee8e4420b07a17d2feb3fcc2
I20260618 13:59:28.325131 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc72a06ee8e4420b07a17d2feb3fcc2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:28.325354 28548 raft_consensus.cc:3060] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.330726 28799 tablet_bootstrap.cc:492] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.330901 28799 ts_tablet_manager.cc:1403] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.046s user 0.022s sys 0.013s
I20260618 13:59:28.331544 28799 raft_consensus.cc:359] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.331736 28799 raft_consensus.cc:385] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.331794 28799 raft_consensus.cc:740] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.332010 28799 consensus_queue.cc:260] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.332455 28799 ts_tablet_manager.cc:1434] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 13:59:28.332823 28799 tablet_bootstrap.cc:492] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.334626 28799 tablet_bootstrap.cc:654] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.336248 29221 raft_consensus.cc:493] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.336445 29221 raft_consensus.cc:515] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.336694 28548 raft_consensus.cc:2468] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:28.337097 29221 leader_election.cc:290] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:28.337124 28618 leader_election.cc:304] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:28.337409 29186 raft_consensus.cc:2804] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.337610 29186 raft_consensus.cc:697] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.337939 29186 consensus_queue.cc:237] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.338588 29301 leader_election.cc:290] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.340576 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b59255b14df4158afe37cb8890e7415" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:28.340682 28483 leader_election.cc:343] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 8b59255b14df4158afe37cb8890e7415
I20260618 13:59:28.340773 28681 raft_consensus.cc:2468] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:28.341189 28484 leader_election.cc:304] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:28.341621 29221 raft_consensus.cc:2804] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.341724 29221 raft_consensus.cc:493] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.341854 29221 raft_consensus.cc:3060] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.342064 28797 tablet_bootstrap.cc:492] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.342281 28797 ts_tablet_manager.cc:1403] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.077s user 0.006s sys 0.001s
I20260618 13:59:28.343248 28797 raft_consensus.cc:359] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.343474 28797 raft_consensus.cc:385] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.343552 28797 raft_consensus.cc:740] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.343851 28797 consensus_queue.cc:260] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.344383 28797 ts_tablet_manager.cc:1434] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260618 13:59:28.344686 28797 tablet_bootstrap.cc:492] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.344951 28799 tablet_bootstrap.cc:492] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.345136 28799 ts_tablet_manager.cc:1403] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.006s
I20260618 13:59:28.345867 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a6764a5e024903848561c9935b1a43" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.345117 29221 raft_consensus.cc:515] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.346001 28799 raft_consensus.cc:359] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.346179 28799 raft_consensus.cc:385] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.346292 28799 raft_consensus.cc:740] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
W20260618 13:59:28.346517 28618 leader_election.cc:343] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.346527 28799 consensus_queue.cc:260] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.346659 28618 leader_election.cc:304] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.347064 28799 ts_tablet_manager.cc:1434] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260618 13:59:28.347460 28483 leader_election.cc:343] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 8b59255b14df4158afe37cb8890e7415
I20260618 13:59:28.347880 29221 leader_election.cc:290] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:28.348004 28799 tablet_bootstrap.cc:492] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.347944 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b59255b14df4158afe37cb8890e7415" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:28.348117 28681 raft_consensus.cc:3060] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.349586 29303 raft_consensus.cc:2749] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.349879 28797 tablet_bootstrap.cc:654] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.351418 28681 raft_consensus.cc:2468] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:28.351565 28799 tablet_bootstrap.cc:654] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.352011 28484 leader_election.cc:304] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:28.352553 29293 raft_consensus.cc:493] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.352766 29293 raft_consensus.cc:515] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.353569 29293 leader_election.cc:290] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.354460 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e95c88e4b0a046a3ad69ee49af47c7c3" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.354914 29221 raft_consensus.cc:2804] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election won for term 1
W20260618 13:59:28.354983 28346 leader_election.cc:343] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.355374 29293 raft_consensus.cc:493] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.355491 29293 raft_consensus.cc:515] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.356026 29293 leader_election.cc:290] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.356637 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0285c75c632457383e81805ab036aaf" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.356762 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0285c75c632457383e81805ab036aaf" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:28.357100 28346 leader_election.cc:343] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.357398 28346 leader_election.cc:343] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.357398 29221 raft_consensus.cc:697] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:28.357532 28346 leader_election.cc:304] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.357795 29221 consensus_queue.cc:237] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.357935 29293 raft_consensus.cc:2749] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.358225 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e95c88e4b0a046a3ad69ee49af47c7c3" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:28.358773 28346 leader_election.cc:343] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.358930 28346 leader_election.cc:304] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.359409 29293 raft_consensus.cc:2749] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.362802 28799 tablet_bootstrap.cc:492] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.364920 28799 ts_tablet_manager.cc:1403] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.003s
W20260618 13:59:28.365556 28616 connection.cc:573] client connection to 127.27.100.60:42083 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260618 13:59:28.365608 28482 connection.cc:573] client connection to 127.27.100.60:42083 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260618 13:59:28.365808 28799 raft_consensus.cc:359] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.366022 28799 raft_consensus.cc:385] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.366115 28799 raft_consensus.cc:740] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.366372 28799 consensus_queue.cc:260] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.367107 28799 ts_tablet_manager.cc:1434] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:28.367470 28799 tablet_bootstrap.cc:492] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.369875 28799 tablet_bootstrap.cc:654] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.371750 29293 raft_consensus.cc:493] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.371922 29293 raft_consensus.cc:515] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.372732 29293 leader_election.cc:290] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:28.373531 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe750fc5d998448caeab0a6c808dfba7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:28.374214 28346 leader_election.cc:343] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.375072 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe750fc5d998448caeab0a6c808dfba7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:28.375694 28346 leader_election.cc:343] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.375826 28346 leader_election.cc:304] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.376223 29293 raft_consensus.cc:2749] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.376564 29293 raft_consensus.cc:493] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.376741 29293 raft_consensus.cc:515] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.377465 29293 leader_election.cc:290] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.377933 29221 raft_consensus.cc:493] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.378085 29221 raft_consensus.cc:515] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.378757 29221 leader_election.cc:290] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.378911 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118259402d7d48dda381cdd18eec0f1b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:28.379437 28346 leader_election.cc:343] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.379594 28484 leader_election.cc:343] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: fca76774f119435899188313a33dad98
I20260618 13:59:28.380002 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118259402d7d48dda381cdd18eec0f1b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:28.380510 28483 leader_election.cc:343] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: fca76774f119435899188313a33dad98
I20260618 13:59:28.380651 28483 leader_election.cc:304] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:28.381080 28073 connection.cc:573] server connection from 127.0.0.1:34172 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260618 13:59:28.381220 28344 connection.cc:573] client connection to 127.27.100.60:42083 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260618 13:59:28.382055 28799 tablet_bootstrap.cc:492] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.382184 28799 ts_tablet_manager.cc:1403] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.004s
I20260618 13:59:28.383173 28799 raft_consensus.cc:359] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.383276 29221 raft_consensus.cc:2749] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.383358 28799 raft_consensus.cc:385] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.383440 28799 raft_consensus.cc:740] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.383708 28799 consensus_queue.cc:260] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:28.383760 28346 leader_election.cc:343] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.383919 29351 negotiation.cc:336] Failed RPC negotiation. Trace:
0618 13:59:28.366619 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.27.100.60:42083 (local address 127.27.100.3:58575)
0618 13:59:28.383524 (+ 16905us) negotiation.cc:107] Waiting for socket to connect
0618 13:59:28.383673 (+ 149us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":16782,"threads_started":1}
I20260618 13:59:28.383936 28346 leader_election.cc:304] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.384328 28799 ts_tablet_manager.cc:1434] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:28.384574 29293 raft_consensus.cc:2749] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.384701 28799 tablet_bootstrap.cc:492] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.386543 28799 tablet_bootstrap.cc:654] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:28.389181 29345 negotiation.cc:336] Failed RPC negotiation. Trace:
0618 13:59:28.336121 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.27.100.60:42083 (local address 127.0.0.1:33632)
0618 13:59:28.363269 (+ 27148us) negotiation.cc:107] Waiting for socket to connect
0618 13:59:28.363290 (+ 21us) client_negotiation.cc:175] Beginning negotiation
0618 13:59:28.363423 (+ 133us) client_negotiation.cc:253] Sending NEGOTIATE NegotiatePB request
0618 13:59:28.385902 (+ 22479us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":26981,"thread_start_us":27784,"threads_started":1}
W20260618 13:59:28.396209 29350 negotiation.cc:336] Failed RPC negotiation. Trace:
0618 13:59:28.366634 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.27.100.60:42083 (local address 127.27.100.2:55921)
0618 13:59:28.395803 (+ 29169us) negotiation.cc:107] Waiting for socket to connect
0618 13:59:28.395971 (+ 168us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":29050,"threads_started":1}
I20260618 13:59:28.402232 28797 tablet_bootstrap.cc:492] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.402396 28797 ts_tablet_manager.cc:1403] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.058s user 0.007s sys 0.002s
I20260618 13:59:28.403504 28797 raft_consensus.cc:359] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.403735 28797 raft_consensus.cc:385] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.403855 28797 raft_consensus.cc:740] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.404209 28797 consensus_queue.cc:260] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.404897 28797 ts_tablet_manager.cc:1434] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:28.407042 28797 tablet_bootstrap.cc:492] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:28.407270 28983 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.60:42083, user_credentials={real_user=slave}} blocked reactor thread for 105972us
I20260618 13:59:28.418206 28797 tablet_bootstrap.cc:654] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.418903 28798 tablet_bootstrap.cc:492] T 1f81428f804a485c967026c4527fd250 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:28.419075 28798 ts_tablet_manager.cc:1403] T 1f81428f804a485c967026c4527fd250 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.099s user 0.011s sys 0.005s
I20260618 13:59:28.420791 29293 raft_consensus.cc:493] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.420966 29293 raft_consensus.cc:515] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.421655 29293 leader_election.cc:290] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.422557 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d502d67826ff4d7a8b0f515e5d8b6d77" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:28.423108 28346 leader_election.cc:343] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.423440 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d502d67826ff4d7a8b0f515e5d8b6d77" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.423612 28798 raft_consensus.cc:359] T 1f81428f804a485c967026c4527fd250 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.423815 28798 raft_consensus.cc:385] T 1f81428f804a485c967026c4527fd250 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260618 13:59:28.424110 28346 leader_election.cc:343] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.424214 28346 leader_election.cc:304] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.424623 29293 raft_consensus.cc:2749] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:28.425402 28983 leader_election.cc:336] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083): Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111)
I20260618 13:59:28.423925 28798 raft_consensus.cc:740] T 1f81428f804a485c967026c4527fd250 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
W20260618 13:59:28.426481 28983 leader_election.cc:336] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083): Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111)
I20260618 13:59:28.426473 28798 consensus_queue.cc:260] T 1f81428f804a485c967026c4527fd250 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.427738 28799 tablet_bootstrap.cc:492] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.427893 28799 ts_tablet_manager.cc:1403] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.043s user 0.013s sys 0.020s
I20260618 13:59:28.428682 28799 raft_consensus.cc:359] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.428893 28799 raft_consensus.cc:385] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.429041 28799 raft_consensus.cc:740] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.429311 28799 consensus_queue.cc:260] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:28.432379 28983 consensus_peers.cc:597] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 -> Peer 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083): Couldn't send request to peer 7637285570574a6dbcef1de6f84ecac0. Status: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:28.433120 28799 ts_tablet_manager.cc:1434] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260618 13:59:28.433508 29293 raft_consensus.cc:493] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.433589 28799 tablet_bootstrap.cc:492] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.433678 29293 raft_consensus.cc:515] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.434245 29293 leader_election.cc:290] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.435111 28798 ts_tablet_manager.cc:1434] T 1f81428f804a485c967026c4527fd250 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.016s user 0.003s sys 0.000s
I20260618 13:59:28.435202 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41107eb1d9164a9c98060003b4027412" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.435380 28799 tablet_bootstrap.cc:654] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.435420 28681 raft_consensus.cc:2468] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:28.435791 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41107eb1d9164a9c98060003b4027412" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.438356 28798 tablet_bootstrap.cc:492] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:28.439494 28346 leader_election.cc:304] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
W20260618 13:59:28.439764 28346 leader_election.cc:343] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.440222 28798 tablet_bootstrap.cc:654] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.440423 29293 raft_consensus.cc:2804] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.440527 29293 raft_consensus.cc:493] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.440619 29293 raft_consensus.cc:3060] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.443810 29293 raft_consensus.cc:515] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.445115 28097 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 2 FOLLOWER]: Refusing update from remote peer a950ab7cd66b4cb0b29592aa692c2dd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 488. Preceding OpId from leader: term: 2 index: 501. (index mismatch)
I20260618 13:59:28.447510 29183 raft_consensus.cc:493] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.447652 29183 raft_consensus.cc:515] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.448186 29183 leader_election.cc:290] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.449496 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc0adedecee4405e86b0a37b9e62e352" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.449949 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc0adedecee4405e86b0a37b9e62e352" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.450619 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41107eb1d9164a9c98060003b4027412" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:28.450811 28681 raft_consensus.cc:3060] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.451879 28799 tablet_bootstrap.cc:492] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.452059 28799 ts_tablet_manager.cc:1403] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.004s
I20260618 13:59:28.453066 28799 raft_consensus.cc:359] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.453328 28799 raft_consensus.cc:385] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.453435 28799 raft_consensus.cc:740] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.453763 28681 raft_consensus.cc:2468] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:28.453751 28799 consensus_queue.cc:260] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.454443 28799 ts_tablet_manager.cc:1434] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:28.454793 28799 tablet_bootstrap.cc:492] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.455152 29282 consensus_queue.cc:1048] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 501, Last known committed idx: 488, Time since last communication: 0.000s
I20260618 13:59:28.457105 28799 tablet_bootstrap.cc:654] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:28.457940 28346 leader_election.cc:343] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.458166 28346 leader_election.cc:304] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
W20260618 13:59:28.458463 28346 leader_election.cc:343] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.458568 28346 leader_election.cc:304] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.459103 29183 raft_consensus.cc:2804] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.469863 28798 tablet_bootstrap.cc:492] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:28.470038 28798 ts_tablet_manager.cc:1403] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.032s user 0.013s sys 0.012s
I20260618 13:59:28.470777 28798 raft_consensus.cc:359] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.470981 28798 raft_consensus.cc:385] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.471051 28798 raft_consensus.cc:740] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.471321 28798 consensus_queue.cc:260] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.472869 28798 ts_tablet_manager.cc:1434] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:28.473196 28798 tablet_bootstrap.cc:492] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:28.474954 28798 tablet_bootstrap.cc:654] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.479018 29221 raft_consensus.cc:493] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.479158 29221 raft_consensus.cc:515] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.479776 29221 leader_election.cc:290] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.480171 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516421cd843b44769efa2b555697c065" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.480404 28681 raft_consensus.cc:2468] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:28.480476 29364 raft_consensus.cc:2749] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.481104 28484 leader_election.cc:304] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:28.481482 29221 raft_consensus.cc:2804] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.481621 29221 raft_consensus.cc:493] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.481720 29221 raft_consensus.cc:3060] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.483583 29293 leader_election.cc:290] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.483816 29293 raft_consensus.cc:493] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.483950 29293 raft_consensus.cc:515] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.484527 29293 leader_election.cc:290] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.484602 29221 raft_consensus.cc:515] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.485536 29299 raft_consensus.cc:493] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260618 13:59:28.481483 28483 leader_election.cc:343] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 516421cd843b44769efa2b555697c065
I20260618 13:59:28.485671 29299 raft_consensus.cc:515] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.486123 29221 leader_election.cc:290] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.486603 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233040a16b0d4b61b977ca57c1afec2d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:28.487121 28618 leader_election.cc:343] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.487113 28483 leader_election.cc:343] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 516421cd843b44769efa2b555697c065
I20260618 13:59:28.487917 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233040a16b0d4b61b977ca57c1afec2d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:28.488131 28409 raft_consensus.cc:2468] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:28.488515 28617 leader_election.cc:304] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.488767 29186 raft_consensus.cc:2804] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.488848 29186 raft_consensus.cc:493] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.488934 29186 raft_consensus.cc:3060] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.488973 29299 leader_election.cc:290] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.489338 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "516421cd843b44769efa2b555697c065" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:28.489496 28681 raft_consensus.cc:3060] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.491757 29186 raft_consensus.cc:515] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.492688 29186 leader_election.cc:290] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.492805 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233040a16b0d4b61b977ca57c1afec2d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:28.492964 28409 raft_consensus.cc:3060] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.495844 28409 raft_consensus.cc:2468] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:28.496222 29364 raft_consensus.cc:493] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.496327 28617 leader_election.cc:304] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:28.496397 29364 raft_consensus.cc:515] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.497112 29364 leader_election.cc:290] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.497296 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233040a16b0d4b61b977ca57c1afec2d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:28.497859 28618 leader_election.cc:343] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.498303 29186 raft_consensus.cc:2804] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.498574 29186 raft_consensus.cc:697] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.498881 29186 consensus_queue.cc:237] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.510689 28797 tablet_bootstrap.cc:492] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.510883 28797 ts_tablet_manager.cc:1403] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.106s user 0.001s sys 0.003s
I20260618 13:59:28.511680 28797 raft_consensus.cc:359] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.511932 28797 raft_consensus.cc:385] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.512013 28797 raft_consensus.cc:740] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.512327 28797 consensus_queue.cc:260] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.512936 28797 ts_tablet_manager.cc:1434] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 13:59:28.513309 28797 tablet_bootstrap.cc:492] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.520044 28797 tablet_bootstrap.cc:654] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.521198 29186 raft_consensus.cc:493] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.521354 29186 raft_consensus.cc:515] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.521991 29186 leader_election.cc:290] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.522975 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6b01b897134d5bbd6a7d93b82ddb5e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:28.523542 28618 leader_election.cc:343] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.523897 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6b01b897134d5bbd6a7d93b82ddb5e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:28.524175 28409 raft_consensus.cc:2468] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:28.524945 28617 leader_election.cc:304] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.525333 29299 raft_consensus.cc:2804] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.525445 29299 raft_consensus.cc:493] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.525532 29299 raft_consensus.cc:3060] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.529294 28681 raft_consensus.cc:2468] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:28.529806 28484 leader_election.cc:304] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:28.530264 29221 raft_consensus.cc:2804] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.530386 29221 raft_consensus.cc:697] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:28.530666 29221 consensus_queue.cc:237] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.530936 28798 tablet_bootstrap.cc:492] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:28.531071 28798 ts_tablet_manager.cc:1403] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.058s user 0.003s sys 0.009s
I20260618 13:59:28.532142 28798 raft_consensus.cc:359] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.532305 28798 raft_consensus.cc:385] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.532414 28798 raft_consensus.cc:740] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.532608 28798 consensus_queue.cc:260] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.533141 28798 ts_tablet_manager.cc:1434] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:28.533494 28798 tablet_bootstrap.cc:492] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:28.535279 28798 tablet_bootstrap.cc:654] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.536789 29299 raft_consensus.cc:515] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.537569 29299 leader_election.cc:290] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.537832 28797 tablet_bootstrap.cc:492] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.537966 28797 ts_tablet_manager.cc:1403] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.003s
I20260618 13:59:28.538228 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6b01b897134d5bbd6a7d93b82ddb5e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:28.538772 28618 leader_election.cc:343] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.538826 28797 raft_consensus.cc:359] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.539007 28797 raft_consensus.cc:385] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.539105 28797 raft_consensus.cc:740] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.539065 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6b01b897134d5bbd6a7d93b82ddb5e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:28.539251 28409 raft_consensus.cc:3060] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.539361 28797 consensus_queue.cc:260] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.539912 28797 ts_tablet_manager.cc:1434] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:28.540918 28797 tablet_bootstrap.cc:492] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.549738 28797 tablet_bootstrap.cc:654] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.550606 28409 raft_consensus.cc:2468] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:28.551797 28798 tablet_bootstrap.cc:492] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:28.551087 28799 tablet_bootstrap.cc:492] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.551973 28798 ts_tablet_manager.cc:1403] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.000s
I20260618 13:59:28.552414 28617 leader_election.cc:304] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.552688 29299 raft_consensus.cc:2804] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.552826 29299 raft_consensus.cc:697] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.552738 28798 raft_consensus.cc:359] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.552942 28798 raft_consensus.cc:385] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.553004 28798 raft_consensus.cc:740] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.553114 29299 consensus_queue.cc:237] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.553267 28798 consensus_queue.cc:260] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.554955 28798 ts_tablet_manager.cc:1434] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260618 13:59:28.555265 28798 tablet_bootstrap.cc:492] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:28.556746 28797 tablet_bootstrap.cc:492] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.556977 28797 ts_tablet_manager.cc:1403] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.001s
I20260618 13:59:28.557789 28797 raft_consensus.cc:359] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.558069 28797 raft_consensus.cc:385] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.558185 28797 raft_consensus.cc:740] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.558498 28797 consensus_queue.cc:260] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.559176 28797 ts_tablet_manager.cc:1434] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:28.559540 28797 tablet_bootstrap.cc:492] T 1829d1b69cdd4e748c5901c8e02cac2e P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.559598 28798 tablet_bootstrap.cc:654] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.561267 28797 tablet_bootstrap.cc:654] T 1829d1b69cdd4e748c5901c8e02cac2e P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.562686 29299 raft_consensus.cc:493] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.562835 29299 raft_consensus.cc:515] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.563364 29299 leader_election.cc:290] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.563966 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b88ed7cf79c4929a8704f66691cbed4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:28.564486 28618 leader_election.cc:343] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.568559 29183 raft_consensus.cc:697] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:28.568817 28799 ts_tablet_manager.cc:1403] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.097s user 0.011s sys 0.051s
I20260618 13:59:28.568869 29362 raft_consensus.cc:493] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.568956 29183 consensus_queue.cc:237] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.569272 28797 tablet_bootstrap.cc:492] T 1829d1b69cdd4e748c5901c8e02cac2e P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.569214 29362 raft_consensus.cc:515] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.569470 29221 raft_consensus.cc:493] T a1a06078974e4bfbbc4b949802a39bf1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.569617 29221 raft_consensus.cc:515] T a1a06078974e4bfbbc4b949802a39bf1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.569720 28799 raft_consensus.cc:359] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.569901 28799 raft_consensus.cc:385] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.569994 28799 raft_consensus.cc:740] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.570200 29221 leader_election.cc:290] T a1a06078974e4bfbbc4b949802a39bf1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.570160 29362 leader_election.cc:290] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.570199 28799 consensus_queue.cc:260] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.570784 28799 ts_tablet_manager.cc:1434] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:28.570792 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1a06078974e4bfbbc4b949802a39bf1" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.571135 28799 tablet_bootstrap.cc:492] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.572755 28797 ts_tablet_manager.cc:1403] T 1829d1b69cdd4e748c5901c8e02cac2e P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.001s
I20260618 13:59:28.572916 28799 tablet_bootstrap.cc:654] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.573473 28797 raft_consensus.cc:359] T 1829d1b69cdd4e748c5901c8e02cac2e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.573625 28797 raft_consensus.cc:385] T 1829d1b69cdd4e748c5901c8e02cac2e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.573700 28797 raft_consensus.cc:740] T 1829d1b69cdd4e748c5901c8e02cac2e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.573940 28797 consensus_queue.cc:260] T 1829d1b69cdd4e748c5901c8e02cac2e P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.574311 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1a06078974e4bfbbc4b949802a39bf1" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:28.574452 28797 ts_tablet_manager.cc:1434] T 1829d1b69cdd4e748c5901c8e02cac2e P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:28.574590 28411 raft_consensus.cc:2393] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:28.574923 29266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader a950ab7cd66b4cb0b29592aa692c2dd0. Latest consensus state: current_term: 2 leader_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:28.575121 29266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [sys.catalog]: This master's current role is: FOLLOWER
I20260618 13:59:28.579540 28797 tablet_bootstrap.cc:492] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.579843 28483 leader_election.cc:304] T a1a06078974e4bfbbc4b949802a39bf1 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.580252 29221 raft_consensus.cc:3060] T a1a06078974e4bfbbc4b949802a39bf1 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.581272 28797 tablet_bootstrap.cc:654] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.586264 29221 raft_consensus.cc:2749] T a1a06078974e4bfbbc4b949802a39bf1 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.589480 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b88ed7cf79c4929a8704f66691cbed4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:28.589800 28409 raft_consensus.cc:2468] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:28.590258 28617 leader_election.cc:304] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.590549 29186 raft_consensus.cc:493] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.590703 29186 raft_consensus.cc:515] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.591288 29186 leader_election.cc:290] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.591459 29186 raft_consensus.cc:2804] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.591534 29186 raft_consensus.cc:493] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.591657 29186 raft_consensus.cc:3060] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.591598 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84189a4b1194c87bbdf81f6e0ed56e4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.591814 28549 raft_consensus.cc:2468] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:28.592623 28618 leader_election.cc:304] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:28.592996 29299 raft_consensus.cc:2804] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.593107 29299 raft_consensus.cc:493] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.593237 29299 raft_consensus.cc:3060] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.594298 29266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:28.594507 29266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [sys.catalog]: This master's current role is: FOLLOWER
I20260618 13:59:28.595100 29186 raft_consensus.cc:515] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.595862 29299 raft_consensus.cc:515] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.596438 29299 leader_election.cc:290] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.597524 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84189a4b1194c87bbdf81f6e0ed56e4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:28.597734 28548 raft_consensus.cc:3060] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.600277 28798 tablet_bootstrap.cc:492] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:28.600433 28798 ts_tablet_manager.cc:1403] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.045s user 0.002s sys 0.002s
I20260618 13:59:28.600457 28548 raft_consensus.cc:2468] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:28.600880 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b88ed7cf79c4929a8704f66691cbed4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:28.601258 28798 raft_consensus.cc:359] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.601423 28798 raft_consensus.cc:385] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.601508 28798 raft_consensus.cc:740] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.601744 28798 consensus_queue.cc:260] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.602804 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b88ed7cf79c4929a8704f66691cbed4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:28.602975 28618 leader_election.cc:304] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:28.603034 28410 raft_consensus.cc:3060] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:28.603271 28618 leader_election.cc:343] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.603580 28617 leader_election.cc:343] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: c84189a4b1194c87bbdf81f6e0ed56e4
W20260618 13:59:28.603904 28617 leader_election.cc:343] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: c84189a4b1194c87bbdf81f6e0ed56e4
I20260618 13:59:28.604212 29299 raft_consensus.cc:493] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.604429 29299 raft_consensus.cc:515] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.605791 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188ebf1727240cd886ba7662f8fcf43" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.605966 28549 raft_consensus.cc:2468] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:28.606195 28410 raft_consensus.cc:2468] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:28.606601 29303 raft_consensus.cc:2804] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.606781 29303 raft_consensus.cc:697] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.607141 29299 leader_election.cc:290] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.610776 28798 ts_tablet_manager.cc:1434] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260618 13:59:28.611093 28798 tablet_bootstrap.cc:492] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:28.612877 28798 tablet_bootstrap.cc:654] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.607061 29303 consensus_queue.cc:237] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.607373 28618 leader_election.cc:304] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:28.607690 28617 leader_election.cc:304] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
W20260618 13:59:28.613585 28617 leader_election.cc:343] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 2188ebf1727240cd886ba7662f8fcf43
I20260618 13:59:28.613906 29301 raft_consensus.cc:2804] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.614055 29301 raft_consensus.cc:493] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.614158 29301 raft_consensus.cc:3060] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.614871 29303 raft_consensus.cc:2804] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.614988 29303 raft_consensus.cc:697] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.615262 29303 consensus_queue.cc:237] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.617537 29301 raft_consensus.cc:515] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.618307 29301 leader_election.cc:290] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.619598 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188ebf1727240cd886ba7662f8fcf43" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:28.619817 28549 raft_consensus.cc:3060] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:28.624560 28617 leader_election.cc:343] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 2188ebf1727240cd886ba7662f8fcf43
I20260618 13:59:28.625138 28549 raft_consensus.cc:2468] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:28.625598 28618 leader_election.cc:304] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:28.626159 29301 raft_consensus.cc:2804] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.626336 29301 raft_consensus.cc:697] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.626698 29301 consensus_queue.cc:237] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.595860 29186 leader_election.cc:290] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.627991 29362 raft_consensus.cc:493] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.628183 29362 raft_consensus.cc:515] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.628795 29362 leader_election.cc:290] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.630246 29303 raft_consensus.cc:493] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.630686 29303 raft_consensus.cc:515] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.631930 29303 leader_election.cc:290] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.632407 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d5e75b4a8247b29d69d07fad5ca5db" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:28.632691 28409 raft_consensus.cc:2468] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:28.633195 28617 leader_election.cc:304] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:28.633500 28797 tablet_bootstrap.cc:492] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.633682 28797 ts_tablet_manager.cc:1403] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.054s user 0.006s sys 0.017s
I20260618 13:59:28.633616 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d5e75b4a8247b29d69d07fad5ca5db" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:28.634212 28618 leader_election.cc:343] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.634697 28797 raft_consensus.cc:359] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.634907 28797 raft_consensus.cc:385] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.635013 28797 raft_consensus.cc:740] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.635291 28797 consensus_queue.cc:260] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.635589 29186 raft_consensus.cc:2804] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.635735 29186 raft_consensus.cc:493] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.635792 28797 ts_tablet_manager.cc:1434] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:28.635854 29186 raft_consensus.cc:3060] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.636663 28797 tablet_bootstrap.cc:492] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.638186 28797 tablet_bootstrap.cc:654] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:28.639698 28346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 84561.8us
I20260618 13:59:28.640805 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f278d0acbb0c4a0ca61640bc142e6335" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.641253 28797 tablet_bootstrap.cc:492] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.641175 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897bf6d6bbee4ad89c296325d94a3e53" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.641420 28797 ts_tablet_manager.cc:1403] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260618 13:59:28.641812 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f278d0acbb0c4a0ca61640bc142e6335" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.642590 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897bf6d6bbee4ad89c296325d94a3e53" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.642495 28797 raft_consensus.cc:359] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.642827 28797 raft_consensus.cc:385] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.642825 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aef7c1f111634c98af046e73fa9ffdeb" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.642995 28797 raft_consensus.cc:740] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.643211 28797 consensus_queue.cc:260] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.643795 28797 ts_tablet_manager.cc:1434] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 13:59:28.643847 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6de1293572f4e75af3417f281b9a142" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.644121 28797 tablet_bootstrap.cc:492] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.643903 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aef7c1f111634c98af046e73fa9ffdeb" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.644974 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6de1293572f4e75af3417f281b9a142" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.645087 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41107eb1d9164a9c98060003b4027412" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:28.645376 29362 raft_consensus.cc:493] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.645592 29362 raft_consensus.cc:515] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.645728 28797 tablet_bootstrap.cc:654] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.646384 29362 leader_election.cc:290] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.647075 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1829d1b69cdd4e748c5901c8e02cac2e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.647270 28682 raft_consensus.cc:2468] T 1829d1b69cdd4e748c5901c8e02cac2e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:28.647653 28797 tablet_bootstrap.cc:492] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.647827 28797 ts_tablet_manager.cc:1403] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
W20260618 13:59:28.648013 28346 leader_election.cc:343] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.648236 28346 leader_election.cc:343] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.648404 28346 leader_election.cc:343] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.648571 28346 leader_election.cc:343] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.648551 28797 raft_consensus.cc:359] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:28.648757 28346 leader_election.cc:343] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.648761 28797 raft_consensus.cc:385] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.648865 28797 raft_consensus.cc:740] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.649113 28797 consensus_queue.cc:260] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.649434 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1829d1b69cdd4e748c5901c8e02cac2e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.649618 28797 ts_tablet_manager.cc:1434] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:28.649909 28797 tablet_bootstrap.cc:492] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.651826 28797 tablet_bootstrap.cc:654] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.653440 28797 tablet_bootstrap.cc:492] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.653589 28797 ts_tablet_manager.cc:1403] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260618 13:59:28.654320 28797 raft_consensus.cc:359] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.654536 28797 raft_consensus.cc:385] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.654650 28797 raft_consensus.cc:740] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.654934 28797 consensus_queue.cc:260] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.655642 28797 ts_tablet_manager.cc:1434] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 13:59:28.656018 28797 tablet_bootstrap.cc:492] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.656996 29186 raft_consensus.cc:515] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.658350 28797 tablet_bootstrap.cc:654] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.658383 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d5e75b4a8247b29d69d07fad5ca5db" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:28.658987 28618 leader_election.cc:343] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.663278 28346 leader_election.cc:343] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.663435 28346 leader_election.cc:304] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.663781 29362 raft_consensus.cc:2749] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:28.664240 28346 leader_election.cc:343] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.664373 28346 leader_election.cc:304] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.664624 29362 raft_consensus.cc:2749] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:28.665057 28346 leader_election.cc:343] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.665187 28346 leader_election.cc:304] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.665510 29362 raft_consensus.cc:2749] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:28.666002 28346 leader_election.cc:343] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.666127 28346 leader_election.cc:304] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.667618 28797 tablet_bootstrap.cc:492] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.667833 28797 ts_tablet_manager.cc:1403] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260618 13:59:28.668902 28797 raft_consensus.cc:359] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.669118 28797 raft_consensus.cc:385] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.669214 28797 raft_consensus.cc:740] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.669502 28797 consensus_queue.cc:260] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.670222 28797 ts_tablet_manager.cc:1434] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:28.670629 28797 tablet_bootstrap.cc:492] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.675897 28797 tablet_bootstrap.cc:654] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.677348 29299 raft_consensus.cc:493] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.677505 29299 raft_consensus.cc:515] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.678655 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fddea8fba4da40d7a5780e6750caf5e9" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.679209 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fddea8fba4da40d7a5780e6750caf5e9" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
W20260618 13:59:28.679327 28618 leader_election.cc:343] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.679497 28410 raft_consensus.cc:2468] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:28.679522 29299 leader_election.cc:290] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.680040 28617 leader_election.cc:304] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.681277 29299 raft_consensus.cc:2804] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.681414 29299 raft_consensus.cc:493] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.681526 29299 raft_consensus.cc:3060] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.685168 28797 tablet_bootstrap.cc:492] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.685360 28797 ts_tablet_manager.cc:1403] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.004s
I20260618 13:59:28.685322 29299 raft_consensus.cc:515] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.686450 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fddea8fba4da40d7a5780e6750caf5e9" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:28.686669 28797 raft_consensus.cc:359] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:28.686993 28618 leader_election.cc:343] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.686596 29303 consensus_queue.cc:1048] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:28.687007 28797 raft_consensus.cc:385] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.687340 28797 raft_consensus.cc:740] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.687695 28797 consensus_queue.cc:260] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.689316 28797 ts_tablet_manager.cc:1434] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260618 13:59:28.690475 28797 tablet_bootstrap.cc:492] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.692833 28797 tablet_bootstrap.cc:654] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.695272 29186 leader_election.cc:290] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.702204 28797 tablet_bootstrap.cc:492] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.702459 28797 ts_tablet_manager.cc:1403] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260618 13:59:28.703567 28797 raft_consensus.cc:359] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.703843 28797 raft_consensus.cc:385] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.703948 28797 raft_consensus.cc:740] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.704965 29299 leader_election.cc:290] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.705745 29362 raft_consensus.cc:2749] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.706203 28798 tablet_bootstrap.cc:492] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:28.706380 28798 ts_tablet_manager.cc:1403] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.095s user 0.026s sys 0.031s
I20260618 13:59:28.706595 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d5e75b4a8247b29d69d07fad5ca5db" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:28.706676 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fddea8fba4da40d7a5780e6750caf5e9" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:28.706856 28410 raft_consensus.cc:3060] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.706910 28408 raft_consensus.cc:3060] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.707319 28798 raft_consensus.cc:359] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.707435 29183 raft_consensus.cc:493] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.707516 28798 raft_consensus.cc:385] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.707581 28798 raft_consensus.cc:740] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.707607 29183 raft_consensus.cc:515] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.707849 28798 consensus_queue.cc:260] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.708633 28798 ts_tablet_manager.cc:1434] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:28.709088 28798 tablet_bootstrap.cc:492] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:28.711257 28798 tablet_bootstrap.cc:654] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.713376 29299 raft_consensus.cc:493] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.713657 29299 raft_consensus.cc:515] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.714857 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a0691de253b4b008e5d995fd6b876c2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:28.715521 28618 leader_election.cc:343] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.704299 28797 consensus_queue.cc:260] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.718724 28797 ts_tablet_manager.cc:1434] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.016s user 0.000s sys 0.002s
I20260618 13:59:28.718711 29395 sys_catalog.cc:455] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:28.719111 29395 sys_catalog.cc:458] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: This master's current role is: LEADER
I20260618 13:59:28.719610 29286 catalog_manager.cc:1510] Loading table and tablet metadata into memory...
I20260618 13:59:28.719903 28797 tablet_bootstrap.cc:492] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.722774 28797 tablet_bootstrap.cc:654] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.726962 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_35260f90f6814fa3915b4280cff96d1b [id=02ffed3d852845d6845fb5176f036fce]
I20260618 13:59:28.727558 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_768d783dc0a7416686369250c3cd855c [id=0321ff97d1674a3891eae8cbba76387a]
I20260618 13:59:28.728053 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_9424eec16a284672a0fe7ab3d0852b40 [id=070ea48af78346aa8ff43f2bb369730c]
I20260618 13:59:28.728921 28410 raft_consensus.cc:2468] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:28.729125 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_6dc717aafef14b22ab95830a6cf50642 [id=0751d5ab69704387856ad7d40431e74b]
I20260618 13:59:28.729526 29301 consensus_queue.cc:1048] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:28.729676 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_0e9f574416d84a0f826fa9a1fcc56210 [id=098fa24bb71e4fe1aecc33aea1275ef0]
I20260618 13:59:28.730393 28617 leader_election.cc:304] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.715811 29299 leader_election.cc:290] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.730767 29299 raft_consensus.cc:2804] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.730931 29299 raft_consensus.cc:697] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.733388 28797 tablet_bootstrap.cc:492] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.733534 28797 ts_tablet_manager.cc:1403] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.003s
I20260618 13:59:28.734328 28797 raft_consensus.cc:359] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.734545 28797 raft_consensus.cc:385] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.734649 28797 raft_consensus.cc:740] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.734918 28797 consensus_queue.cc:260] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:28.735962 28346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.3:35503, user_credentials={real_user=slave}} blocked reactor thread for 48937.1us
I20260618 13:59:28.735994 29183 leader_election.cc:290] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.736285 28408 raft_consensus.cc:2468] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:28.736316 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a0691de253b4b008e5d995fd6b876c2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:28.735623 28797 ts_tablet_manager.cc:1434] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:28.736548 28410 raft_consensus.cc:2468] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:28.736797 28797 tablet_bootstrap.cc:492] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.736815 28799 tablet_bootstrap.cc:492] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.736979 28799 ts_tablet_manager.cc:1403] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.166s user 0.001s sys 0.007s
I20260618 13:59:28.736820 28617 leader_election.cc:304] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.737375 28617 leader_election.cc:304] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.737665 29303 raft_consensus.cc:493] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.737883 29303 raft_consensus.cc:515] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.738194 29186 raft_consensus.cc:2804] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.738965 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "751cd9bd961b4f56a588ed02a05f0212" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.740103 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_7ae70108b1714e5ca374cc653bed7b33 [id=0cf9d7859b7d4d95bf769070c43003d3]
I20260618 13:59:28.740151 28799 raft_consensus.cc:359] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.740382 28799 raft_consensus.cc:385] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.740495 28799 raft_consensus.cc:740] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.740777 28799 consensus_queue.cc:260] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.741317 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_19cfaeff1d174399b77939349481eb5a [id=0cfca3de151f4100b6003fa7b76f0e97]
I20260618 13:59:28.741431 28799 ts_tablet_manager.cc:1434] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.000s sys 0.000s
I20260618 13:59:28.741773 28799 tablet_bootstrap.cc:492] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.741784 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_8783a8d8c526428281436b62b2581a33 [id=0d83fc68255c458587390c58b12e00d7]
I20260618 13:59:28.742240 28346 leader_election.cc:304] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:28.742591 29183 raft_consensus.cc:2804] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.742709 29183 raft_consensus.cc:493] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.742801 29183 raft_consensus.cc:3060] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.745456 29183 raft_consensus.cc:515] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.745992 29183 leader_election.cc:290] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.746838 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd06bb7cffc34412b420d70435fce373" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:28.746874 28346 leader_election.cc:343] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.747298 29183 raft_consensus.cc:493] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.739356 29301 raft_consensus.cc:2804] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.748816 29301 raft_consensus.cc:493] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260618 13:59:28.739573 28618 leader_election.cc:343] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.748970 29301 raft_consensus.cc:3060] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.749271 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_259b4dd1855b42a8824f00306cb5223a [id=15c34c0db3b54b27ac29f13880f49f0d]
I20260618 13:59:28.750252 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_8c9c9a6eea174bd68d8fb9d9ebaaf603 [id=185e7792e79644a2aced2b949fca635b]
I20260618 13:59:28.751085 29221 raft_consensus.cc:493] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.751245 29221 raft_consensus.cc:515] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.750882 29303 leader_election.cc:290] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.751901 29221 leader_election.cc:290] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.740585 28797 tablet_bootstrap.cc:654] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.752158 29301 raft_consensus.cc:515] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.752820 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a01dfb4d59549399c94640f16045555" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.753028 28681 raft_consensus.cc:2468] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:28.743296 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd06bb7cffc34412b420d70435fce373" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.753497 28484 leader_election.cc:304] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:28.753549 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a0691de253b4b008e5d995fd6b876c2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:28.742679 29299 consensus_queue.cc:237] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.750981 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_0fbace1c1aa746a6a2a57c878631218b [id=18b5971926ed4fff9048003a51d3a4fa]
I20260618 13:59:28.753805 29221 raft_consensus.cc:2804] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.753906 29221 raft_consensus.cc:493] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.753981 29221 raft_consensus.cc:3060] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.754513 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15]
I20260618 13:59:28.755177 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_ddb8f9243a5d47d1be63b3a371ca17ac [id=225463c2467047edae05b73a3132638f]
I20260618 13:59:28.755220 29183 raft_consensus.cc:515] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.755309 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1829d1b69cdd4e748c5901c8e02cac2e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:28.755478 28682 raft_consensus.cc:3060] T 1829d1b69cdd4e748c5901c8e02cac2e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.755592 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_f37d21a9a59249d6a0b87c3788ff4873 [id=22eb9037fc66472793e9fd3f03fd7218]
I20260618 13:59:28.755841 29183 leader_election.cc:290] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.756827 29221 raft_consensus.cc:515] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.757311 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_659a3d495ed44be4bfb2193816362e25 [id=2659803e822d4761a15e0e89644cc472]
I20260618 13:59:28.757716 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_b5800aca80594ec1bd3a86ec47dbfc64 [id=2710734517a24ad6baf0d47bf23e9bdf]
I20260618 13:59:28.758073 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_c35470a550534fa2ad072f932e981df5 [id=2797c7fefad74c1e82ab31fb9ad4db70]
I20260618 13:59:28.758220 29183 raft_consensus.cc:493] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.758491 28682 raft_consensus.cc:2468] T 1829d1b69cdd4e748c5901c8e02cac2e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:28.758643 29230 raft_consensus.cc:493] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.758623 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a01dfb4d59549399c94640f16045555" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:28.758813 28681 raft_consensus.cc:3060] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.758790 29230 raft_consensus.cc:515] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:28.758925 28483 leader_election.cc:343] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 4a01dfb4d59549399c94640f16045555
W20260618 13:59:28.758597 28618 leader_election.cc:343] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.759385 29230 leader_election.cc:290] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.758386 29183 raft_consensus.cc:515] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.760026 29183 leader_election.cc:290] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:28.760046 29221 leader_election.cc:290] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:28.760715 28483 leader_election.cc:343] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 0a51e830e3014726b1d1c2608973f036
W20260618 13:59:28.761093 28484 leader_election.cc:343] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 0a51e830e3014726b1d1c2608973f036
W20260618 13:59:28.761174 28483 leader_election.cc:343] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 4a01dfb4d59549399c94640f16045555
I20260618 13:59:28.761220 28484 leader_election.cc:304] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.761281 28681 raft_consensus.cc:2468] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:28.761955 28484 leader_election.cc:304] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:28.762446 28798 tablet_bootstrap.cc:492] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:28.762398 29230 raft_consensus.cc:2749] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:28.762403 28346 leader_election.cc:343] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.762637 29230 raft_consensus.cc:2804] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.762666 28346 leader_election.cc:304] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:28.762669 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1829d1b69cdd4e748c5901c8e02cac2e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:28.748648 29186 raft_consensus.cc:697] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.762814 29230 raft_consensus.cc:697] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:28.763065 29230 consensus_queue.cc:237] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.763103 29186 consensus_queue.cc:237] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.763875 29183 raft_consensus.cc:2804] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.764130 29183 raft_consensus.cc:697] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:28.764429 29183 consensus_queue.cc:237] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.766191 28798 ts_tablet_manager.cc:1403] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.057s user 0.007s sys 0.003s
I20260618 13:59:28.767517 28798 raft_consensus.cc:359] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.767964 28798 raft_consensus.cc:385] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.768162 28798 raft_consensus.cc:740] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.768667 28798 consensus_queue.cc:260] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.769635 29301 leader_election.cc:290] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.769872 28798 ts_tablet_manager.cc:1434] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260618 13:59:28.770495 28798 tablet_bootstrap.cc:492] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
W20260618 13:59:28.771812 28346 leader_election.cc:343] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.771962 28346 leader_election.cc:304] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.772475 28797 tablet_bootstrap.cc:492] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.772492 28799 tablet_bootstrap.cc:654] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.772720 28797 ts_tablet_manager.cc:1403] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.036s user 0.004s sys 0.002s
I20260618 13:59:28.773627 28797 raft_consensus.cc:359] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.773973 28797 raft_consensus.cc:385] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.774343 28797 raft_consensus.cc:740] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.774786 28797 consensus_queue.cc:260] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.775393 28797 ts_tablet_manager.cc:1434] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:28.776520 29362 raft_consensus.cc:2749] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:28.776958 28346 leader_election.cc:343] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.778007 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b06ae7444a48dbbec9e096fd7e4052" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.778203 29362 raft_consensus.cc:493] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.778338 29362 raft_consensus.cc:515] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.778959 29362 leader_election.cc:290] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.780453 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "751cd9bd961b4f56a588ed02a05f0212" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:28.780634 28409 raft_consensus.cc:2468] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:28.781678 28617 leader_election.cc:304] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.782233 29299 raft_consensus.cc:2804] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.782675 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a0691de253b4b008e5d995fd6b876c2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:28.782882 28409 raft_consensus.cc:3060] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.783665 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_b0e3e2341f764392af42b949d03dd1f6 [id=27b6a19a28f24f2b8a758bbff06312d0]
I20260618 13:59:28.784292 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_ab57b782a92747e78e2bfe28480aa0ec [id=28d43fd5660f459a9acfab2fc0dfbdb1]
I20260618 13:59:28.785406 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_e97ff5cd68d24da7a9a803e10b35dbe6 [id=2c1b442aeecf44ee8b6140cacdb37627]
I20260618 13:59:28.785607 29186 raft_consensus.cc:493] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.785784 29186 raft_consensus.cc:515] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.785960 28409 raft_consensus.cc:2468] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:28.786077 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_7cd820b0428a41ada19a2f14560a67e4 [id=2c73a9d87a914f03b09ef422d49f7a95]
I20260618 13:59:28.786608 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_dc44ae6b83d844b6a16ce10ba6dce0ad [id=2e7a199c805d4453b2137b5fa69d0b4e]
I20260618 13:59:28.786870 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c17e76ae9b44207998a43e3576908c4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.787176 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_0a85d79af65f48a3803bfd1248593a11 [id=2f4d62be772a421ab777fd23be58b719]
W20260618 13:59:28.787456 28618 leader_election.cc:343] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.787425 28617 leader_election.cc:304] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.787889 29186 leader_election.cc:290] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.788064 29186 raft_consensus.cc:2804] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.788077 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_efb2b8e451084b7ba0de07e0a6e538c2 [id=333c0fd3837b4d89a9197eb6a826c311]
I20260618 13:59:28.788198 29186 raft_consensus.cc:697] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.788470 29186 consensus_queue.cc:237] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.788545 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c17e76ae9b44207998a43e3576908c4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:28.788720 28408 raft_consensus.cc:2468] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:28.788908 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_9763de0aea1d44d785311a303abb1d27 [id=34c58ef63c21476685cbc59915bbfd0a]
I20260618 13:59:28.789433 28617 leader_election.cc:304] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.789680 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_031e1a03030b45c99c1270d612008f91 [id=387d3ff1762749efad926181fe4c746a]
I20260618 13:59:28.782492 29299 raft_consensus.cc:493] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.790107 29299 raft_consensus.cc:3060] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.790198 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c]
I20260618 13:59:28.790681 29303 raft_consensus.cc:2804] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.790791 29303 raft_consensus.cc:493] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.790920 29303 raft_consensus.cc:3060] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.791698 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_0ec7ef7f4c6e48d5b196e45f78a4fbf2 [id=3b33c0ca00d248298402d6bcec792be4]
I20260618 13:59:28.792169 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_7699960fdee24242bf51bcfcbd6f68c2 [id=3b9ab803536344d3a59ff77b2b58ae31]
I20260618 13:59:28.792970 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]
I20260618 13:59:28.793455 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_a44222ec49e54499bbd3d2c256abc71c [id=3f176321fcf44af2b5a4103634b1908f]
I20260618 13:59:28.793923 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_9c526031302044508f50fead286b4465 [id=408b312e0fad42ae8fb0ab84abd85c0b]
I20260618 13:59:28.794025 29303 raft_consensus.cc:515] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.794950 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_21633c55ca8c4614a2a0b241a0b89ee1 [id=411f4d0d27f94aa9842287ba693b2a01]
I20260618 13:59:28.795423 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_7cea29656b76480fb24d915e7478121a [id=41e4176c7e17448a87ba484c4b447fe8]
I20260618 13:59:28.795440 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c17e76ae9b44207998a43e3576908c4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:28.795756 28798 tablet_bootstrap.cc:654] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.795982 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_7910db46cf394cdebe608e7efb3388cc [id=421bf4faf8174ccd88dc15799671c74d]
I20260618 13:59:28.796011 28797 tablet_bootstrap.cc:492] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:28.796218 28618 leader_election.cc:343] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.796649 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c17e76ae9b44207998a43e3576908c4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:28.796725 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_97b6bb39b3bf4612be391b9dd78d1646 [id=42a2a24590834abc9537669aaf223a7c]
I20260618 13:59:28.796852 28408 raft_consensus.cc:3060] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.797747 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_dbc6162c06f74e71b792bd1f7b2388fe [id=49576dabdecf41aca1b15ff3fc0c113c]
I20260618 13:59:28.798247 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]
I20260618 13:59:28.798754 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_697444611a2643bfb1fe30fc545c47f3 [id=4aece4c39f65473494e823b598e81bd5]
I20260618 13:59:28.798950 29299 raft_consensus.cc:515] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.799319 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_abe180e115a04052ae89cc92d3c1c1e6 [id=4cbcd521f63047a38f5a9b9def56df7f]
I20260618 13:59:28.799639 28408 raft_consensus.cc:2468] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
W20260618 13:59:28.800145 28346 leader_election.cc:343] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.800480 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b8f273def3e4b7caa4d624cf87a106c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.800606 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b06ae7444a48dbbec9e096fd7e4052" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.800874 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46bfe8cc2078414d84e65e22ed4ef8a2" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.800892 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46bfe8cc2078414d84e65e22ed4ef8a2" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.800942 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b8f273def3e4b7caa4d624cf87a106c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.803815 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_c528008fd94b4fc8b7341f5b23dff76b [id=516d3679e0e84ab28429e59b6895db4e]
I20260618 13:59:28.804471 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_76724583c0db40378d257d673a601220 [id=5278d8eb60e4448ebae7afc3edb0db8e]
W20260618 13:59:28.804708 28346 leader_election.cc:343] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.804966 28346 leader_election.cc:343] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.805074 28346 leader_election.cc:304] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.805105 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]
W20260618 13:59:28.805524 28346 leader_election.cc:343] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.805738 28346 leader_election.cc:343] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.805879 28346 leader_election.cc:304] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.806216 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_9a78338c3e414b0abea1281082202f57 [id=5697846ccd9c4c659755a0f9b8a66b72]
I20260618 13:59:28.806967 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_4108c7ea79f5426395746346c44df521 [id=5a04afb74af64b17914f1587d7ddc542]
I20260618 13:59:28.808208 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]
I20260618 13:59:28.809134 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_3ef2ff183a6044e2a9e069a4eac133b8 [id=60e91b2138bf41a5a8dee65c1874678a]
I20260618 13:59:28.810020 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_93b6c79a288040ceae924af14f939c35 [id=61ce711b673b44189a507d1abf26bd75]
I20260618 13:59:28.810873 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_b5feee88c56b42b6bd1b8ecb81a134bb [id=635bd8a93b22415fab4ee952247c6870]
I20260618 13:59:28.811044 28617 leader_election.cc:304] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.811479 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_7e0256d5218f469c9fef22ba76bb4e64 [id=65db0daf3f084f09bfd0bba3011741d9]
I20260618 13:59:28.811410 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "751cd9bd961b4f56a588ed02a05f0212" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:28.811518 29401 raft_consensus.cc:493] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.812493 29362 raft_consensus.cc:2749] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:28.812011 28618 leader_election.cc:343] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.812748 29362 raft_consensus.cc:2749] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.812817 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_2a3bcaddd3034d67a5ea7c78f0ea9e7c [id=6c30b030a60b47e2be48dfe6e9cfdfdb]
I20260618 13:59:28.812958 29401 raft_consensus.cc:515] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.813341 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_e8ae99dfd8e84d7bb2723eb5fe129c2f [id=6d8f3597814241f3bc8c920d3a59008e]
I20260618 13:59:28.814033 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_b8741a7fd29f4e78a948b1c419595a85 [id=6f4d908593ae40f09e2c94cef218e1fd]
I20260618 13:59:28.815569 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_62bf5ce52fb14bd089678663dd009a91 [id=713af0f85a434ed0bfb791c48f75dc9d]
I20260618 13:59:28.815532 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c2b924a096b479cad31c29bf7c99172" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.815758 28548 raft_consensus.cc:2468] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:28.816452 28798 tablet_bootstrap.cc:492] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:28.816732 28798 ts_tablet_manager.cc:1403] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.046s user 0.000s sys 0.006s
I20260618 13:59:28.817618 28618 leader_election.cc:304] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:28.817934 29186 raft_consensus.cc:2804] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.818080 29186 raft_consensus.cc:493] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.818212 29186 raft_consensus.cc:3060] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.818235 28798 raft_consensus.cc:359] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.818562 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_5d485acde41448d984aa79fa58b1866b [id=726324d365b94ff6bac24030222ca34f]
I20260618 13:59:28.818632 28798 raft_consensus.cc:385] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.818783 28798 raft_consensus.cc:740] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.819396 28798 consensus_queue.cc:260] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.820964 29186 raft_consensus.cc:515] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.821482 28798 ts_tablet_manager.cc:1434] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:28.822405 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c2b924a096b479cad31c29bf7c99172" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:28.822603 28798 tablet_bootstrap.cc:492] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:28.822654 28548 raft_consensus.cc:3060] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.825489 28548 raft_consensus.cc:2468] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:28.825987 28618 leader_election.cc:304] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:28.827548 29401 leader_election.cc:290] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.827723 29401 raft_consensus.cc:2804] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.827854 29401 raft_consensus.cc:697] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.828104 29401 consensus_queue.cc:237] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:28.828409 28617 leader_election.cc:343] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 4c2b924a096b479cad31c29bf7c99172
I20260618 13:59:28.829568 28798 tablet_bootstrap.cc:654] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:28.829980 28346 leader_election.cc:343] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.830132 28346 leader_election.cc:304] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.830569 29362 raft_consensus.cc:493] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.830732 29362 raft_consensus.cc:515] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.830910 29303 leader_election.cc:290] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.831295 29362 leader_election.cc:290] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:28.831460 29362 raft_consensus.cc:2749] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.832659 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_dabd3b5bd3ff4784b839d502baec0c6e [id=79633bfcdf584058b406f418caa25aea]
I20260618 13:59:28.833336 29186 leader_election.cc:290] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.833492 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_f958c4ce58c34bfda98454664e202af2 [id=7b747ef3b2494ca78c499c5bb75648d4]
I20260618 13:59:28.833999 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_6f1f355b18a34ac09304992c96d40765 [id=7bb817b09d154696977f3d68d9461d4e]
I20260618 13:59:28.834734 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_c99584ecc88948a097b39c22d68b3938 [id=7c39b3c432e944bca05f7b2070a79ff9]
W20260618 13:59:28.835269 28617 leader_election.cc:343] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 4c2b924a096b479cad31c29bf7c99172
I20260618 13:59:28.836184 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_26e7d8c4f91144268a2a43e83992667d [id=803d5220ce1a4dcfaf27d93c574be2ac]
I20260618 13:59:28.813654 29402 raft_consensus.cc:2804] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.814271 28797 tablet_bootstrap.cc:654] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.839568 28799 tablet_bootstrap.cc:492] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.839872 28799 ts_tablet_manager.cc:1403] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.098s user 0.001s sys 0.002s
I20260618 13:59:28.839874 29402 raft_consensus.cc:697] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.841153 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece7909d056c4ba8a5dca6423f0c1781" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.841114 29402 consensus_queue.cc:237] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.841281 28799 raft_consensus.cc:359] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.841457 28799 raft_consensus.cc:385] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.841528 28799 raft_consensus.cc:740] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.841759 28799 consensus_queue.cc:260] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:28.842247 28346 leader_election.cc:343] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.842445 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece7909d056c4ba8a5dca6423f0c1781" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.845487 28799 ts_tablet_manager.cc:1434] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260618 13:59:28.845940 28799 tablet_bootstrap.cc:492] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.850075 29362 raft_consensus.cc:493] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.850262 29362 raft_consensus.cc:515] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.850955 29362 leader_election.cc:290] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.851902 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_12d6cc95071949c6a7a297cd08b5f898 [id=840c933184144aebb3659d939f679f00]
I20260618 13:59:28.852278 29299 leader_election.cc:290] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.852716 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "751cd9bd961b4f56a588ed02a05f0212" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:28.852959 28409 raft_consensus.cc:3060] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.855772 28409 raft_consensus.cc:2468] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:28.856249 28617 leader_election.cc:304] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.856961 29299 raft_consensus.cc:2804] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.857178 29299 raft_consensus.cc:697] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.857312 28799 tablet_bootstrap.cc:654] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.857564 29299 consensus_queue.cc:237] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.858484 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_e601511ed59d407b86040479045e84ec [id=8721763b01de4d54980bae1f82434853]
I20260618 13:59:28.859032 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_122b5d1531834ee2aa6bbc6593efb740 [id=873af388ffca420d958310ef2b50795c]
W20260618 13:59:28.859597 28346 leader_election.cc:343] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.859741 28346 leader_election.cc:304] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.859998 29362 raft_consensus.cc:2749] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.860174 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_bbf208e6e39a47349ab50198f178974e [id=8e4cf344c21640728c419a084f14c7b1]
I20260618 13:59:28.860534 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512a9e63ba324a30b8d8c37e585ebdf8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.861341 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_38a3b325a9514a3d829c6ba288f9673a [id=8f0879e0c0b74edfa8ae99d11cda07a1]
I20260618 13:59:28.861943 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_c70bc1c98b3b49d7aa3080904667732d [id=901874cb80e74ba589679bb7b95c8fe4]
I20260618 13:59:28.862412 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_7fa3da42543d4978aac5ac383ed10056 [id=9102540068234d1993fa164035dabc6e]
I20260618 13:59:28.862741 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512a9e63ba324a30b8d8c37e585ebdf8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.863170 28797 tablet_bootstrap.cc:492] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.863344 28797 ts_tablet_manager.cc:1403] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.088s user 0.004s sys 0.005s
I20260618 13:59:28.864284 28797 raft_consensus.cc:359] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.864473 28797 raft_consensus.cc:385] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.864420 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_7e5d58ad3e8043919b66e8890819535c [id=96aff34b68c1494f9228a11eb38de4c7]
I20260618 13:59:28.864575 28797 raft_consensus.cc:740] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.864809 28797 consensus_queue.cc:260] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.865487 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]
W20260618 13:59:28.868270 28346 leader_election.cc:343] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.868510 28346 leader_election.cc:343] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.869513 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_571f6cccbfa3470a83b5542f128f2370 [id=9806bb85160042b095abdbcf04cf3748]
I20260618 13:59:28.876012 29362 raft_consensus.cc:493] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.876216 29362 raft_consensus.cc:515] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.876905 29362 leader_election.cc:290] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:28.877251 29401 raft_consensus.cc:493] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.876012 28346 leader_election.cc:304] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.877477 29401 raft_consensus.cc:515] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.878263 29401 leader_election.cc:290] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.878481 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_bf5ce6c6e0b64002994fb45e76ab30eb [id=a0e6904d88704db39c194c34ac9a7faf]
I20260618 13:59:28.879855 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_fac3fdd9b466497683bc00300a85fed1 [id=a4ac38531af34b4681abba96b42ffb7f]
I20260618 13:59:28.881047 28797 ts_tablet_manager.cc:1434] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260618 13:59:28.881430 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b16f7d3c7837498e996bba1818110642" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:28.881714 28409 raft_consensus.cc:2468] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:28.882148 28617 leader_election.cc:304] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:28.882612 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b16f7d3c7837498e996bba1818110642" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.883000 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_caa35bd6fefa4c5bab90b4b7f1ec48d9 [id=a508a0cb72474a748aacdfd918e3782a]
W20260618 13:59:28.883220 28618 leader_election.cc:343] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.883920 29362 raft_consensus.cc:2749] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.884307 28797 tablet_bootstrap.cc:492] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.884975 28799 tablet_bootstrap.cc:492] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:28.884912 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc36fbea853a4d4e9f61909200239aed" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.885200 28799 ts_tablet_manager.cc:1403] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.039s user 0.005s sys 0.001s
I20260618 13:59:28.885789 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc36fbea853a4d4e9f61909200239aed" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.886073 28799 raft_consensus.cc:359] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.886284 28797 tablet_bootstrap.cc:654] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.886332 28799 raft_consensus.cc:385] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.886469 28799 raft_consensus.cc:740] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.886680 28799 consensus_queue.cc:260] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:28.889035 28346 leader_election.cc:343] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:28.889451 28346 leader_election.cc:343] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.889493 28799 ts_tablet_manager.cc:1434] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260618 13:59:28.889635 28346 leader_election.cc:304] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.889919 29362 raft_consensus.cc:2749] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.890094 28799 tablet_bootstrap.cc:492] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.890084 29401 raft_consensus.cc:2804] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.890251 29401 raft_consensus.cc:493] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.890503 29401 raft_consensus.cc:3060] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.891353 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_bfcd14e3d49e4de09270ed85ec9cde4c [id=aa0452c5d31844c4b99cd207c0163445]
I20260618 13:59:28.897631 28799 tablet_bootstrap.cc:654] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.898705 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]
I20260618 13:59:28.901904 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]
I20260618 13:59:28.902508 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_9a10a9aa97574e7c89586d1e4e863171 [id=af1859c644b44c1d831314d27a494f94]
I20260618 13:59:28.903946 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_d31dc25d118d4cd2bb4dd8f034752827 [id=b1d85ea3aa7047d285ab7c86da8213fd]
I20260618 13:59:28.904443 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]
I20260618 13:59:28.904945 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_83d021ed148243ffb453cb5e2484b7b4 [id=b3cae943ecdf410ca1cf711c5ddfe14a]
I20260618 13:59:28.905635 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_8f22977cfa824da99508b4063a05f8e6 [id=b527fd0afd76403f94cb5fe9f4c95094]
I20260618 13:59:28.906323 29401 raft_consensus.cc:515] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.906896 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_706eec6806a94d3f8d5b60708a0c1cda [id=b817db93d50144d7b7c238550eb53b10]
I20260618 13:59:28.907563 28798 tablet_bootstrap.cc:492] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:28.907620 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b16f7d3c7837498e996bba1818110642" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:28.907960 28798 ts_tablet_manager.cc:1403] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.086s user 0.017s sys 0.011s
I20260618 13:59:28.908346 28799 tablet_bootstrap.cc:492] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
W20260618 13:59:28.908324 28618 leader_election.cc:343] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.908510 28799 ts_tablet_manager.cc:1403] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.000s
I20260618 13:59:28.909364 28799 raft_consensus.cc:359] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.909515 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b16f7d3c7837498e996bba1818110642" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:28.909581 28799 raft_consensus.cc:385] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.909667 28799 raft_consensus.cc:740] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.909694 28409 raft_consensus.cc:3060] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.909915 28799 consensus_queue.cc:260] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.909832 28798 raft_consensus.cc:359] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.911530 28798 raft_consensus.cc:385] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.911919 28798 raft_consensus.cc:740] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.912904 28798 consensus_queue.cc:260] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.913081 28409 raft_consensus.cc:2468] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:28.913497 28798 ts_tablet_manager.cc:1434] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260618 13:59:28.913823 28798 tablet_bootstrap.cc:492] T 9a0691de253b4b008e5d995fd6b876c2 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:28.918582 28617 leader_election.cc:304] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.913100 28799 ts_tablet_manager.cc:1434] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:28.918918 29301 raft_consensus.cc:2804] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:28.919305 29301 raft_consensus.cc:697] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:28.919471 28799 tablet_bootstrap.cc:492] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:28.919584 29301 consensus_queue.cc:237] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.920630 28798 tablet_bootstrap.cc:654] T 9a0691de253b4b008e5d995fd6b876c2 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.921365 28799 tablet_bootstrap.cc:654] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:28.921958 28617 consensus_peers.cc:597] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 864f2602840c40fc8606b3784e671cba. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:28.922725 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_e24e443c392040b0adf1e8c8eb581a10 [id=b97c43b351b946c48f356b1ea1134239]
I20260618 13:59:28.923411 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_7d317fa28e3142a19d15c8360151c357 [id=bba3ab8821ca48a5aa1104094d78892a]
I20260618 13:59:28.923826 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_8e7925d0e7eb421685f09b8c53667351 [id=bbe67818f204445c89ede107910a3bdd]
I20260618 13:59:28.924232 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_d838306823ff4e22a68efe0a8cb751ff [id=bbfd0a6a47db4b1b8c534405b1068d5e]
I20260618 13:59:28.925251 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]
I20260618 13:59:28.925658 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_40fdb9a7e5ad4ff2bf879ec4e02a4b8d [id=c1e1069f4a1640e2a3f86b302f1dcc7a]
I20260618 13:59:28.926800 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]
I20260618 13:59:28.927789 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]
I20260618 13:59:28.928339 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_2f8e4182bb594c2bad77994c12ce821f [id=c70ab3da89b74f9ba473123d260c0f14]
I20260618 13:59:28.929040 29230 raft_consensus.cc:493] T 1f81428f804a485c967026c4527fd250 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.929198 29230 raft_consensus.cc:515] T 1f81428f804a485c967026c4527fd250 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.929953 29230 leader_election.cc:290] T 1f81428f804a485c967026c4527fd250 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.930393 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f81428f804a485c967026c4527fd250" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.931067 28798 tablet_bootstrap.cc:492] T 9a0691de253b4b008e5d995fd6b876c2 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:28.931241 28798 ts_tablet_manager.cc:1403] T 9a0691de253b4b008e5d995fd6b876c2 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.002s
I20260618 13:59:28.931372 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f81428f804a485c967026c4527fd250" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:28.932053 28798 raft_consensus.cc:359] T 9a0691de253b4b008e5d995fd6b876c2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.932247 28798 raft_consensus.cc:385] T 9a0691de253b4b008e5d995fd6b876c2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.932315 28798 raft_consensus.cc:740] T 9a0691de253b4b008e5d995fd6b876c2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.932559 28798 consensus_queue.cc:260] T 9a0691de253b4b008e5d995fd6b876c2 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.933259 28798 ts_tablet_manager.cc:1434] T 9a0691de253b4b008e5d995fd6b876c2 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 13:59:28.933911 28798 tablet_bootstrap.cc:492] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:28.935400 28409 raft_consensus.cc:2393] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:28.936532 28798 tablet_bootstrap.cc:654] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.937171 28483 leader_election.cc:304] T 1f81428f804a485c967026c4527fd250 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.938185 29230 raft_consensus.cc:3060] T 1f81428f804a485c967026c4527fd250 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.941947 28797 tablet_bootstrap.cc:492] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.942124 28797 ts_tablet_manager.cc:1403] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.058s user 0.010s sys 0.002s
I20260618 13:59:28.942831 29301 raft_consensus.cc:493] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.942946 28797 raft_consensus.cc:359] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.943158 28797 raft_consensus.cc:385] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.943228 28797 raft_consensus.cc:740] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.943509 28797 consensus_queue.cc:260] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.943661 29230 raft_consensus.cc:2749] T 1f81428f804a485c967026c4527fd250 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.942991 29301 raft_consensus.cc:515] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.944543 29301 leader_election.cc:290] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.945192 28797 ts_tablet_manager.cc:1434] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260618 13:59:28.946440 29362 raft_consensus.cc:493] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.946604 29362 raft_consensus.cc:515] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:28.947290 29362 leader_election.cc:290] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:28.947721 28797 tablet_bootstrap.cc:492] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.948336 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc0adedecee4405e86b0a37b9e62e352" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.928896 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_663ed02889aa45938d7d1316d4d19c06 [id=c79cfc8300a94a18873df52a1d83ada3]
I20260618 13:59:28.949141 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc0adedecee4405e86b0a37b9e62e352" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
W20260618 13:59:28.949193 28618 leader_election.cc:343] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.949440 28409 raft_consensus.cc:2468] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:28.949940 28617 leader_election.cc:304] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:28.950431 29301 raft_consensus.cc:2804] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.950555 29301 raft_consensus.cc:493] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.950698 28797 tablet_bootstrap.cc:654] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.950675 29301 raft_consensus.cc:3060] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.954391 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_fd6ef45aeec84b48b8fb7a54db4b4f81 [id=ca38f91f52db4c69a0fb3bc6b8f5cadc]
I20260618 13:59:28.954985 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_334d2fffc20b4a5688939d1067192640 [id=ca499a8d88084635a1bc0c1107ce4b86]
I20260618 13:59:28.955063 29301 raft_consensus.cc:515] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.955621 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_f7b8e1561d67469b9a3709475d661de9 [id=caa2a3299e864159b261740eecc0f64d]
I20260618 13:59:28.955658 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea4bc13eabc4dd98b4f96723498014c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.956089 29301 leader_election.cc:290] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.956995 28797 tablet_bootstrap.cc:492] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.957123 28797 ts_tablet_manager.cc:1403] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.004s
I20260618 13:59:28.957231 28798 tablet_bootstrap.cc:492] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:28.957378 28798 ts_tablet_manager.cc:1403] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.001s
I20260618 13:59:28.957849 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc0adedecee4405e86b0a37b9e62e352" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:28.957942 28797 raft_consensus.cc:359] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.958086 28797 raft_consensus.cc:385] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.958222 28797 raft_consensus.cc:740] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.958125 28798 raft_consensus.cc:359] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:28.958568 28618 leader_election.cc:343] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.958686 28798 raft_consensus.cc:385] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.958792 28798 raft_consensus.cc:740] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.958909 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc0adedecee4405e86b0a37b9e62e352" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:28.959028 28798 consensus_queue.cc:260] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.959591 28798 ts_tablet_manager.cc:1434] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:28.960716 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]
I20260618 13:59:28.960757 28409 raft_consensus.cc:3060] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:28.961462 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_5159a242b12044fc9a0b3e535ea40050 [id=cfb4ca8c4e884ccc8602be52d0b39100]
I20260618 13:59:28.961885 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_74fc7c21dc664ff8bfe0d25aee399bdb [id=d224150fd3a340fc96e75c0b96146b11]
I20260618 13:59:28.962302 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_387aa217e7f04d2f826dc1f82c256a97 [id=d5294ec922b94ef7b1a4394f00462ff4]
I20260618 13:59:28.962965 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_72e5809d8cce4f3bbe8227efcb8e2b2c [id=dcd71daa60ee430d9ca46a1463d9a1bf]
I20260618 13:59:28.963387 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_ea7b830f69254dc49cbc9e3c17c3be85 [id=de842622055849ceac0bac11fbcba6e8]
I20260618 13:59:28.963991 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_a7ff69763fdb4ff3b26b180bf05907d9 [id=e39e2c5718794a748361d7b453c65404]
I20260618 13:59:28.964614 28797 consensus_queue.cc:260] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.965123 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea4bc13eabc4dd98b4f96723498014c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:28.965556 29301 raft_consensus.cc:493] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.965766 29301 raft_consensus.cc:515] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.966552 28797 ts_tablet_manager.cc:1434] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260618 13:59:28.966881 28797 tablet_bootstrap.cc:492] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.968061 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d667ea5f029f46d89fcb3be03069b97b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:28.968242 28408 raft_consensus.cc:2468] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:28.968704 28617 leader_election.cc:304] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
W20260618 13:59:28.968858 28346 leader_election.cc:343] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.968973 29299 raft_consensus.cc:2804] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:28.969061 29299 raft_consensus.cc:493] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:28.969192 29299 raft_consensus.cc:3060] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:28.969288 28346 leader_election.cc:343] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.969707 28346 leader_election.cc:304] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:28.970130 29362 raft_consensus.cc:2749] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:28.971308 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d667ea5f029f46d89fcb3be03069b97b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:28.971480 28798 tablet_bootstrap.cc:492] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:28.971947 29299 raft_consensus.cc:515] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.972704 29299 leader_election.cc:290] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.973388 28798 tablet_bootstrap.cc:654] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.973359 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d667ea5f029f46d89fcb3be03069b97b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:28.973449 28797 tablet_bootstrap.cc:654] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.973522 28408 raft_consensus.cc:3060] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:28.974064 28618 leader_election.cc:343] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.971874 29301 leader_election.cc:290] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:28.976953 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d667ea5f029f46d89fcb3be03069b97b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:28.977995 28618 leader_election.cc:343] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.979080 28797 tablet_bootstrap.cc:492] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:28.981133 28797 ts_tablet_manager.cc:1403] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260618 13:59:28.981942 28797 raft_consensus.cc:359] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.982147 28797 raft_consensus.cc:385] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.982236 28797 raft_consensus.cc:740] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.982520 28797 consensus_queue.cc:260] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.984824 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_aeeec1571f064ab1816cbf26635190fd [id=e408f407250f4c0eb56e1abe07fb6b02]
I20260618 13:59:28.986304 29301 raft_consensus.cc:493] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:28.986516 29301 raft_consensus.cc:515] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:28.987663 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283523e5b22843ba935eabe6283d3a99" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:28.987807 29301 leader_election.cc:290] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:28.987879 28410 raft_consensus.cc:2468] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:28.988024 28798 tablet_bootstrap.cc:492] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:28.988201 28798 ts_tablet_manager.cc:1403] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20260618 13:59:28.988348 28617 leader_election.cc:304] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:28.988556 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_3eaa556cd32f4cc1a275c041a0ca104b [id=e6529022f4a2497ea0ed450a84825b93]
I20260618 13:59:28.989012 28798 raft_consensus.cc:359] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.989228 28798 raft_consensus.cc:385] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:28.989250 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_0a074f8e7c5443959918f5effc28e3d7 [id=e78c685d531d4d1b831e33f2b059b90c]
I20260618 13:59:28.989311 28798 raft_consensus.cc:740] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:28.989578 28798 consensus_queue.cc:260] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:28.989933 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_c6e2f443374a47d2a787049499a0b168 [id=e875c5c4a0004e8287426403b0b9c626]
I20260618 13:59:28.990208 28798 ts_tablet_manager.cc:1434] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:28.990602 28798 tablet_bootstrap.cc:492] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:28.992094 28797 ts_tablet_manager.cc:1434] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260618 13:59:28.992460 28798 tablet_bootstrap.cc:654] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.993705 28797 tablet_bootstrap.cc:492] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:28.994554 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283523e5b22843ba935eabe6283d3a99" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:28.995182 28618 leader_election.cc:343] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:28.996272 28797 tablet_bootstrap.cc:654] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:28.998790 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_dfe94c1f858644c98dba7db12d019477 [id=ea67f187738d41c993153f788e016e73]
I20260618 13:59:28.999743 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_84c59c9ab3f54fb4930771088e194560 [id=ee8aa366b2d1461099cb328367424ebd]
I20260618 13:59:29.001017 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]
I20260618 13:59:29.001122 29301 raft_consensus.cc:2804] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.001236 29301 raft_consensus.cc:493] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.001348 29301 raft_consensus.cc:3060] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.003836 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_9cf9f72ae27b44bc9aa43303a7103214 [id=f50e59ae00344758833392841b7f4985]
I20260618 13:59:29.004918 29362 raft_consensus.cc:493] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.005072 29362 raft_consensus.cc:515] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.005247 28798 tablet_bootstrap.cc:492] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:29.005709 29362 leader_election.cc:290] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.005465 28798 ts_tablet_manager.cc:1403] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.002s
I20260618 13:59:29.006297 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827cc0987bd541759e334fb620f04ed5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.007395 28798 raft_consensus.cc:359] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.007666 28798 raft_consensus.cc:385] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.007620 29301 raft_consensus.cc:515] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.007802 28798 raft_consensus.cc:740] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.008131 28798 consensus_queue.cc:260] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.008315 29301 leader_election.cc:290] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.008292 29299 consensus_queue.cc:1048] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.008447 28797 tablet_bootstrap.cc:492] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:29.008572 28797 ts_tablet_manager.cc:1403] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.003s
I20260618 13:59:29.009225 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283523e5b22843ba935eabe6283d3a99" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.009409 28410 raft_consensus.cc:3060] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.009837 28797 raft_consensus.cc:359] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.010003 28797 raft_consensus.cc:385] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.010103 28797 raft_consensus.cc:740] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.010483 28798 ts_tablet_manager.cc:1434] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:29.011497 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]
I20260618 13:59:29.011592 28798 tablet_bootstrap.cc:492] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:29.012166 28797 consensus_queue.cc:260] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.013669 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]
W20260618 13:59:29.015424 28617 consensus_peers.cc:597] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6871232860b41d489e34d95eb897241. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:29.015645 28798 tablet_bootstrap.cc:654] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.015692 29299 raft_consensus.cc:493] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.015952 29299 raft_consensus.cc:515] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.016633 29299 leader_election.cc:290] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.015488 28797 ts_tablet_manager.cc:1434] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260618 13:59:29.017066 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316f470ea82f42fca48e4e96d49c9574" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.017290 28411 raft_consensus.cc:2468] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:29.017803 28617 leader_election.cc:304] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.018075 29299 raft_consensus.cc:2804] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.018188 29299 raft_consensus.cc:493] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.018271 29299 raft_consensus.cc:3060] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.018749 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316f470ea82f42fca48e4e96d49c9574" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.018945 28549 raft_consensus.cc:2468] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:29.020089 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283523e5b22843ba935eabe6283d3a99" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:29.020797 28618 leader_election.cc:343] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.021751 29299 raft_consensus.cc:515] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.022723 29299 leader_election.cc:290] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.023030 28797 tablet_bootstrap.cc:492] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:29.023182 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316f470ea82f42fca48e4e96d49c9574" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.023340 28411 raft_consensus.cc:3060] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.023782 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316f470ea82f42fca48e4e96d49c9574" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:29.024029 28548 raft_consensus.cc:3060] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.024973 28797 tablet_bootstrap.cc:654] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.026392 29286 catalog_manager.cc:679] Loaded metadata for table default.Table_a8705c5cd49a40049ff1020a01aa7f5b [id=f9ff809bb5da4d048e7520cec3316f6d]
I20260618 13:59:29.028636 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827cc0987bd541759e334fb620f04ed5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.028785 28548 raft_consensus.cc:2468] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
W20260618 13:59:29.029371 28346 leader_election.cc:343] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:29.029923 28346 leader_election.cc:343] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.030386 28618 leader_election.cc:304] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.030839 29299 raft_consensus.cc:2804] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.031045 29299 raft_consensus.cc:697] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.031411 29299 consensus_queue.cc:237] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.033039 29230 raft_consensus.cc:493] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.033278 29230 raft_consensus.cc:515] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.034998 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41107eb1d9164a9c98060003b4027412" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.035308 28681 raft_consensus.cc:2393] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate 6fc808af0e884ed8836561665d8e0e7b in this term.
I20260618 13:59:29.035630 29230 leader_election.cc:290] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.034709 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41107eb1d9164a9c98060003b4027412" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.042320 28483 leader_election.cc:304] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.042950 29230 raft_consensus.cc:3060] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.044690 28798 tablet_bootstrap.cc:492] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:29.044947 28798 ts_tablet_manager.cc:1403] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.034s user 0.005s sys 0.020s
I20260618 13:59:29.046245 28798 raft_consensus.cc:359] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.046557 28798 raft_consensus.cc:385] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.046717 28798 raft_consensus.cc:740] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.047080 29230 raft_consensus.cc:2749] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.047163 28798 consensus_queue.cc:260] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.047334 28346 leader_election.cc:304] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.047891 29362 raft_consensus.cc:2749] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.048156 28798 ts_tablet_manager.cc:1434] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260618 13:59:29.048600 28798 tablet_bootstrap.cc:492] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:29.051398 28798 tablet_bootstrap.cc:654] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:29.051852 28617 consensus_peers.cc:597] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 36b8a46a41e543c58a946b6764618c7d. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:29.053670 29230 raft_consensus.cc:493] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.054064 29230 raft_consensus.cc:515] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.054411 29401 leader_election.cc:290] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.055343 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c17e76ae9b44207998a43e3576908c4" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.055668 29230 leader_election.cc:290] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.055732 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c17e76ae9b44207998a43e3576908c4" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.055955 28407 raft_consensus.cc:2393] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:29.056320 28483 leader_election.cc:304] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.056789 29230 raft_consensus.cc:3060] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.058648 28408 raft_consensus.cc:2468] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.060005 29230 raft_consensus.cc:2749] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.061375 28411 raft_consensus.cc:2468] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.061568 28410 raft_consensus.cc:2468] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.061982 28617 leader_election.cc:304] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.062460 29401 raft_consensus.cc:2804] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.062686 28617 leader_election.cc:304] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.062971 29402 raft_consensus.cc:2804] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.063030 29401 raft_consensus.cc:697] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.063138 29402 raft_consensus.cc:697] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.063172 28409 raft_consensus.cc:2468] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.063392 29402 consensus_queue.cc:237] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.063501 29401 consensus_queue.cc:237] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.063609 28617 leader_election.cc:304] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.063920 29186 raft_consensus.cc:2804] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.063941 29286 tablet_loader.cc:96] loaded metadata for tablet 015494374645469785f1f978eaf0e9a5 (table default.Table_7699960fdee24242bf51bcfcbd6f68c2 [id=3b9ab803536344d3a59ff77b2b58ae31])
I20260618 13:59:29.064059 29186 raft_consensus.cc:697] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.064347 29186 consensus_queue.cc:237] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.069216 29286 tablet_loader.cc:96] loaded metadata for tablet 054a7afb3b644aa49df78302cf8f02c9 (table default.Table_2f8e4182bb594c2bad77994c12ce821f [id=c70ab3da89b74f9ba473123d260c0f14])
I20260618 13:59:29.071723 29362 raft_consensus.cc:493] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.071887 29362 raft_consensus.cc:515] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.072583 29362 leader_election.cc:290] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.073215 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3dbf13544ce408ab97834aec5862390" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.073403 29362 raft_consensus.cc:493] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.073590 29362 raft_consensus.cc:515] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:29.074214 28346 leader_election.cc:343] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.074542 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3dbf13544ce408ab97834aec5862390" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:29.075157 28346 leader_election.cc:343] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.075282 28346 leader_election.cc:304] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.075529 29183 raft_consensus.cc:2749] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.075759 29362 leader_election.cc:290] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:29.075929 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a42aa779ae452e9420d730bf8d7d55" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.076634 29362 raft_consensus.cc:493] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.076804 29362 raft_consensus.cc:515] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.077524 29362 leader_election.cc:290] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.080314 29286 tablet_loader.cc:96] loaded metadata for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6])
I20260618 13:59:29.080626 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a42aa779ae452e9420d730bf8d7d55" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.081569 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4144b1b5ea4e968672616bc22bead8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.082620 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4144b1b5ea4e968672616bc22bead8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:29.083186 28346 leader_election.cc:343] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:29.083464 28346 leader_election.cc:343] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.084123 29286 tablet_loader.cc:96] loaded metadata for tablet 118259402d7d48dda381cdd18eec0f1b (table default.Table_7cea29656b76480fb24d915e7478121a [id=41e4176c7e17448a87ba484c4b447fe8])
W20260618 13:59:29.084123 28346 leader_election.cc:343] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.084256 28346 leader_election.cc:304] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.084558 29362 raft_consensus.cc:2749] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:29.084729 28346 leader_election.cc:343] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.084870 28346 leader_election.cc:304] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.085142 29362 raft_consensus.cc:2749] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.087947 29230 raft_consensus.cc:493] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.088106 29230 raft_consensus.cc:515] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.088579 29286 tablet_loader.cc:96] loaded metadata for tablet 15180aefba0544329daee74308ef5010 (table default.Table_97b6bb39b3bf4612be391b9dd78d1646 [id=42a2a24590834abc9537669aaf223a7c])
I20260618 13:59:29.088792 29230 leader_election.cc:290] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.089041 29286 tablet_loader.cc:96] loaded metadata for tablet 15483938a06e459789f87c6bb456bd15 (table default.Table_f37d21a9a59249d6a0b87c3788ff4873 [id=22eb9037fc66472793e9fd3f03fd7218])
I20260618 13:59:29.089854 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c552d1316e3340f4bcfa39618ae309b8" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.090054 28681 raft_consensus.cc:2468] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:29.090620 28484 leader_election.cc:304] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.091053 29230 raft_consensus.cc:2804] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.091150 29230 raft_consensus.cc:493] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.091266 29230 raft_consensus.cc:3060] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.094208 29266 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a950ab7cd66b4cb0b29592aa692c2dd0)
I20260618 13:59:29.094381 29266 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:29.094313 29230 raft_consensus.cc:515] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.095091 29266 leader_election.cc:290] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a950ab7cd66b4cb0b29592aa692c2dd0 (127.27.100.61:45967), 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083)
I20260618 13:59:29.095539 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c552d1316e3340f4bcfa39618ae309b8" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:29.095702 28681 raft_consensus.cc:3060] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.096164 28798 tablet_bootstrap.cc:492] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:29.096318 28798 ts_tablet_manager.cc:1403] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.048s user 0.008s sys 0.005s
W20260618 13:59:29.096673 28483 leader_election.cc:343] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: c552d1316e3340f4bcfa39618ae309b8
I20260618 13:59:29.096845 29230 leader_election.cc:290] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:29.097271 28483 leader_election.cc:343] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: c552d1316e3340f4bcfa39618ae309b8
I20260618 13:59:29.097237 28798 raft_consensus.cc:359] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.097410 28798 raft_consensus.cc:385] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.097506 28798 raft_consensus.cc:740] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.097723 28798 consensus_queue.cc:260] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.098237 28798 ts_tablet_manager.cc:1434] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:29.098373 28681 raft_consensus.cc:2468] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:29.098551 28798 tablet_bootstrap.cc:492] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:29.098857 28484 leader_election.cc:304] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:29.099145 29230 raft_consensus.cc:2804] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.099426 29230 raft_consensus.cc:697] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:29.099767 29230 consensus_queue.cc:237] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.100677 29286 tablet_loader.cc:96] loaded metadata for tablet 1829d1b69cdd4e748c5901c8e02cac2e (table default.Table_e601511ed59d407b86040479045e84ec [id=8721763b01de4d54980bae1f82434853])
I20260618 13:59:29.102124 29286 tablet_loader.cc:96] loaded metadata for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3])
I20260618 13:59:29.103161 29186 consensus_queue.cc:1048] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.107899 28798 tablet_bootstrap.cc:654] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.110203 29286 tablet_loader.cc:96] loaded metadata for tablet 1f81428f804a485c967026c4527fd250 (table default.Table_706eec6806a94d3f8d5b60708a0c1cda [id=b817db93d50144d7b7c238550eb53b10])
I20260618 13:59:29.111761 29286 tablet_loader.cc:96] loaded metadata for tablet 2d1073b726ef4cbeac6c874fc8863e90 (table default.Table_bbf208e6e39a47349ab50198f178974e [id=8e4cf344c21640728c419a084f14c7b1])
I20260618 13:59:29.112171 29286 tablet_loader.cc:96] loaded metadata for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_caa35bd6fefa4c5bab90b4b7f1ec48d9 [id=a508a0cb72474a748aacdfd918e3782a])
I20260618 13:59:29.114609 29186 raft_consensus.cc:493] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.114840 29186 raft_consensus.cc:515] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.115702 29186 leader_election.cc:290] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.116575 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897bf6d6bbee4ad89c296325d94a3e53" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.117063 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897bf6d6bbee4ad89c296325d94a3e53" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.117313 28409 raft_consensus.cc:2468] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:29.118219 28618 leader_election.cc:343] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.118669 28617 leader_election.cc:304] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.119292 29186 raft_consensus.cc:2804] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.121981 29286 tablet_loader.cc:96] loaded metadata for tablet 30f4bd3f86484caab48c1f0806299460 (table default.Table_6dc717aafef14b22ab95830a6cf50642 [id=0751d5ab69704387856ad7d40431e74b])
I20260618 13:59:29.123582 29286 tablet_loader.cc:96] loaded metadata for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522])
I20260618 13:59:29.125160 29362 raft_consensus.cc:493] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.125349 29362 raft_consensus.cc:515] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.125512 28799 tablet_bootstrap.cc:492] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:29.125648 28799 ts_tablet_manager.cc:1403] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.206s user 0.018s sys 0.022s
I20260618 13:59:29.126013 29362 leader_election.cc:290] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.126524 28799 raft_consensus.cc:359] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.126740 28799 raft_consensus.cc:385] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.126865 28799 raft_consensus.cc:740] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.127094 28799 consensus_queue.cc:260] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.127635 28799 ts_tablet_manager.cc:1434] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:29.130676 28799 tablet_bootstrap.cc:492] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:29.130766 28071 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111) [suppressed 7 similar messages]
I20260618 13:59:29.132587 28799 tablet_bootstrap.cc:654] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.133517 29401 raft_consensus.cc:493] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.134939 29186 raft_consensus.cc:493] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.135316 29401 raft_consensus.cc:515] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.135914 29401 leader_election.cc:290] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.136196 29186 raft_consensus.cc:3060] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.136445 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9fa9675234db2abff694a517c0ea8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.137100 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "092a9aba9c6e48f5be9cf5d55ba93678" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.137447 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "092a9aba9c6e48f5be9cf5d55ba93678" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.137660 28409 raft_consensus.cc:2468] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:29.138190 29230 consensus_queue.cc:1048] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:29.138293 28618 leader_election.cc:343] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.138500 29401 consensus_queue.cc:1048] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260618 13:59:29.138589 28617 leader_election.cc:304] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.138885 29401 raft_consensus.cc:2804] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.138990 29401 raft_consensus.cc:493] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.138983 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9fa9675234db2abff694a517c0ea8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.139076 29401 raft_consensus.cc:3060] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.133651 29286 tablet_loader.cc:96] loaded metadata for tablet 340f6bbf18eb412ca3afec237ff7752d (table default.Table_3eaa556cd32f4cc1a275c041a0ca104b [id=e6529022f4a2497ea0ed450a84825b93])
I20260618 13:59:29.140921 28798 tablet_bootstrap.cc:492] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:29.141077 28798 ts_tablet_manager.cc:1403] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.043s user 0.010s sys 0.005s
I20260618 13:59:29.141817 28798 raft_consensus.cc:359] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.141940 29402 consensus_queue.cc:1048] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260618 13:59:29.142012 28798 raft_consensus.cc:385] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.142079 28798 raft_consensus.cc:740] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.142336 28798 consensus_queue.cc:260] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.144630 28798 ts_tablet_manager.cc:1434] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260618 13:59:29.146938 29221 consensus_queue.cc:1048] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:29.149397 28617 consensus_peers.cc:597] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d9c00003300246ada2a356e3e1cf684b. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:29.150844 29401 raft_consensus.cc:515] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.151830 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "092a9aba9c6e48f5be9cf5d55ba93678" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.152012 28409 raft_consensus.cc:3060] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.154551 29186 raft_consensus.cc:515] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.154621 29413 raft_consensus.cc:493] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.154814 29413 raft_consensus.cc:515] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.156270 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897bf6d6bbee4ad89c296325d94a3e53" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:29.157269 29186 leader_election.cc:290] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.157933 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "092a9aba9c6e48f5be9cf5d55ba93678" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:29.155633 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897bf6d6bbee4ad89c296325d94a3e53" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.158438 28410 raft_consensus.cc:3060] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:29.154320 28346 leader_election.cc:343] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.159466 29286 tablet_loader.cc:96] loaded metadata for tablet 35d8dd2c388e43a0be380aa953ec2c82 (table default.Table_a8705c5cd49a40049ff1020a01aa7f5b [id=f9ff809bb5da4d048e7520cec3316f6d])
I20260618 13:59:29.156085 28409 raft_consensus.cc:2468] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.161801 28410 raft_consensus.cc:2468] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.163288 29413 leader_election.cc:290] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.163604 28617 leader_election.cc:304] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.164235 29413 raft_consensus.cc:2804] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.164196 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0285c75c632457383e81805ab036aaf" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.164435 28410 raft_consensus.cc:2468] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:29.164682 29413 raft_consensus.cc:697] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.164913 29401 leader_election.cc:290] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:29.165009 28618 leader_election.cc:343] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.165766 28617 leader_election.cc:304] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.166079 29402 raft_consensus.cc:2804] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.166101 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0285c75c632457383e81805ab036aaf" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.166473 29413 consensus_queue.cc:237] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:29.166966 28346 leader_election.cc:343] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.167773 29186 raft_consensus.cc:493] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.167910 29186 raft_consensus.cc:515] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.168581 29186 leader_election.cc:290] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:29.168735 28618 leader_election.cc:343] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.169222 28617 leader_election.cc:304] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.169283 29402 raft_consensus.cc:697] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.169664 29402 consensus_queue.cc:237] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:29.170250 28618 leader_election.cc:343] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.171093 28797 tablet_bootstrap.cc:492] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:29.171281 28797 ts_tablet_manager.cc:1403] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.154s user 0.002s sys 0.024s
I20260618 13:59:29.171506 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd2385e6752d49429422fc854539e9c6" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.171821 28410 raft_consensus.cc:2468] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:29.172061 28797 raft_consensus.cc:359] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.172297 28797 raft_consensus.cc:385] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.172380 28797 raft_consensus.cc:740] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.172348 28617 leader_election.cc:304] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.172684 29401 raft_consensus.cc:2804] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.172631 28797 consensus_queue.cc:260] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.172760 29401 raft_consensus.cc:493] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.172861 29401 raft_consensus.cc:3060] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.173148 28797 ts_tablet_manager.cc:1434] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:29.173480 28797 tablet_bootstrap.cc:492] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:29.171093 29186 raft_consensus.cc:2804] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.174575 29186 raft_consensus.cc:493] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.174665 29186 raft_consensus.cc:3060] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.176158 29401 raft_consensus.cc:515] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.177244 29401 leader_election.cc:290] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.177359 29186 raft_consensus.cc:515] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.178052 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd2385e6752d49429422fc854539e9c6" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:29.178647 29186 leader_election.cc:290] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:29.178869 28618 leader_election.cc:343] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.178946 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd2385e6752d49429422fc854539e9c6" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:29.179446 28618 leader_election.cc:343] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.180270 28797 tablet_bootstrap.cc:654] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.181650 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0285c75c632457383e81805ab036aaf" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.181787 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0285c75c632457383e81805ab036aaf" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:29.181908 28409 raft_consensus.cc:3060] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:29.182283 28618 leader_election.cc:343] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.184185 28798 tablet_bootstrap.cc:492] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:29.184715 28409 raft_consensus.cc:2468] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.185269 28617 leader_election.cc:304] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.185704 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd2385e6752d49429422fc854539e9c6" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.185886 28410 raft_consensus.cc:3060] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.186023 28798 tablet_bootstrap.cc:654] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.187616 29186 raft_consensus.cc:2804] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.188024 29186 raft_consensus.cc:697] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.188539 29186 consensus_queue.cc:237] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.189129 28410 raft_consensus.cc:2468] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.189541 28617 leader_election.cc:304] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.189890 29400 raft_consensus.cc:2804] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.190028 29400 raft_consensus.cc:697] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.190299 29400 consensus_queue.cc:237] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.196393 29221 consensus_queue.cc:1048] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.203431 29456 raft_consensus.cc:493] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.203616 29456 raft_consensus.cc:515] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.205051 28799 tablet_bootstrap.cc:492] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:29.205253 28799 ts_tablet_manager.cc:1403] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.075s user 0.023s sys 0.022s
I20260618 13:59:29.206148 28799 raft_consensus.cc:359] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.206390 28799 raft_consensus.cc:385] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.206492 28799 raft_consensus.cc:740] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.206768 28799 consensus_queue.cc:260] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.207336 28799 ts_tablet_manager.cc:1434] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:29.207686 28799 tablet_bootstrap.cc:492] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:29.209646 28799 tablet_bootstrap.cc:654] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:29.211419 28483 consensus_peers.cc:597] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: da4d4f03911f4ac099ecc2fa1d8af494. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:29.211853 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a6764a5e024903848561c9935b1a43" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.212057 28682 raft_consensus.cc:2468] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:29.212244 29456 leader_election.cc:290] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.212635 28484 leader_election.cc:304] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
W20260618 13:59:29.212942 28483 leader_election.cc:343] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 37a6764a5e024903848561c9935b1a43
I20260618 13:59:29.213037 29456 raft_consensus.cc:2804] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.213164 29456 raft_consensus.cc:493] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.213282 29456 raft_consensus.cc:3060] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.216159 29456 raft_consensus.cc:515] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.217185 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a6764a5e024903848561c9935b1a43" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:29.217352 28682 raft_consensus.cc:3060] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:29.218700 28071 leader_election.cc:336] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083): Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111)
I20260618 13:59:29.220043 29299 consensus_queue.cc:1048] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.220546 28682 raft_consensus.cc:2468] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:29.220943 28484 leader_election.cc:304] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.221216 29221 raft_consensus.cc:2804] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.221372 29221 raft_consensus.cc:697] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:29.221496 29456 leader_election.cc:290] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.221689 29221 consensus_queue.cc:237] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:29.222411 28483 leader_election.cc:343] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 37a6764a5e024903848561c9935b1a43
I20260618 13:59:29.168285 28346 leader_election.cc:304] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.226775 29362 raft_consensus.cc:2749] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:29.227360 28346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.3:35503, user_credentials={real_user=slave}} blocked reactor thread for 67281.5us
W20260618 13:59:29.228948 28483 consensus_peers.cc:597] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8b59255b14df4158afe37cb8890e7415. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:29.231474 29362 raft_consensus.cc:493] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.231689 29362 raft_consensus.cc:515] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.232419 29362 leader_election.cc:290] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:29.242540 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72fd63e8245d4042b091593881ba11b6" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.243932 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72fd63e8245d4042b091593881ba11b6" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:29.245127 28346 leader_election.cc:343] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:29.245911 28346 leader_election.cc:343] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.246194 28346 leader_election.cc:304] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.246654 29362 raft_consensus.cc:2749] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.254926 29221 raft_consensus.cc:493] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.255120 29221 raft_consensus.cc:515] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.256125 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b9c0754beb4e0e8ed3a7c1e9f368a9" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.256361 28683 raft_consensus.cc:2468] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:29.258610 29221 leader_election.cc:290] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:29.259166 28483 leader_election.cc:343] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 75b9c0754beb4e0e8ed3a7c1e9f368a9
I20260618 13:59:29.261808 28484 leader_election.cc:304] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
W20260618 13:59:29.262821 28483 consensus_peers.cc:597] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eead369de58f40378254697cbbbc6958. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:29.263774 29286 tablet_loader.cc:96] loaded metadata for tablet 39b06ae7444a48dbbec9e096fd7e4052 (table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15])
I20260618 13:59:29.265816 28799 tablet_bootstrap.cc:492] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:29.265969 28799 ts_tablet_manager.cc:1403] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.058s user 0.006s sys 0.009s
I20260618 13:59:29.266206 29456 raft_consensus.cc:2804] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.266342 29456 raft_consensus.cc:493] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.266477 29456 raft_consensus.cc:3060] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.267141 29286 tablet_loader.cc:96] loaded metadata for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_abe180e115a04052ae89cc92d3c1c1e6 [id=4cbcd521f63047a38f5a9b9def56df7f])
I20260618 13:59:29.269378 29286 tablet_loader.cc:96] loaded metadata for tablet 3dd0eaa64271489d92aa380df877f257 (table default.Table_fd6ef45aeec84b48b8fb7a54db4b4f81 [id=ca38f91f52db4c69a0fb3bc6b8f5cadc])
I20260618 13:59:29.269629 28799 raft_consensus.cc:359] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.269840 28799 raft_consensus.cc:385] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.269927 28799 raft_consensus.cc:740] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.270047 29286 tablet_loader.cc:96] loaded metadata for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48])
I20260618 13:59:29.270181 28799 consensus_queue.cc:260] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.270790 28799 ts_tablet_manager.cc:1434] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260618 13:59:29.271128 28799 tablet_bootstrap.cc:492] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:29.273110 28799 tablet_bootstrap.cc:654] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.273070 29286 tablet_loader.cc:96] loaded metadata for tablet 41c92a594cd64982a8972e4a019d1cd8 (table default.Table_387aa217e7f04d2f826dc1f82c256a97 [id=d5294ec922b94ef7b1a4394f00462ff4])
I20260618 13:59:29.275193 29286 tablet_loader.cc:96] loaded metadata for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664])
I20260618 13:59:29.276012 29286 tablet_loader.cc:96] loaded metadata for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_ea7b830f69254dc49cbc9e3c17c3be85 [id=de842622055849ceac0bac11fbcba6e8])
I20260618 13:59:29.276831 29286 tablet_loader.cc:96] loaded metadata for tablet 46bfe8cc2078414d84e65e22ed4ef8a2 (table default.Table_659a3d495ed44be4bfb2193816362e25 [id=2659803e822d4761a15e0e89644cc472])
I20260618 13:59:29.278208 29286 tablet_loader.cc:96] loaded metadata for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3])
I20260618 13:59:29.279906 29413 consensus_queue.cc:1048] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.280303 29286 tablet_loader.cc:96] loaded metadata for tablet 4e4c6537144d425f81fd1fafcd034b77 (table default.Table_122b5d1531834ee2aa6bbc6593efb740 [id=873af388ffca420d958310ef2b50795c])
I20260618 13:59:29.290191 29286 tablet_loader.cc:96] loaded metadata for tablet 512a9e63ba324a30b8d8c37e585ebdf8 (table default.Table_c70bc1c98b3b49d7aa3080904667732d [id=901874cb80e74ba589679bb7b95c8fe4])
I20260618 13:59:29.295372 29286 tablet_loader.cc:96] loaded metadata for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307])
I20260618 13:59:29.293504 29456 raft_consensus.cc:515] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.297482 29456 leader_election.cc:290] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:29.298180 28483 leader_election.cc:343] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 75b9c0754beb4e0e8ed3a7c1e9f368a9
I20260618 13:59:29.298403 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b9c0754beb4e0e8ed3a7c1e9f368a9" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:29.298648 28683 raft_consensus.cc:3060] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.301412 28683 raft_consensus.cc:2468] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:29.301816 28484 leader_election.cc:304] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:29.302356 29458 raft_consensus.cc:2804] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.302524 29458 raft_consensus.cc:697] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:29.302812 29458 consensus_queue.cc:237] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.305739 29221 raft_consensus.cc:493] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.305886 29221 raft_consensus.cc:515] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.305980 29402 consensus_queue.cc:1048] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.306509 29221 leader_election.cc:290] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.307179 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233040a16b0d4b61b977ca57c1afec2d" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.307420 28409 raft_consensus.cc:2393] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:29.308352 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233040a16b0d4b61b977ca57c1afec2d" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.309036 28484 leader_election.cc:304] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.309551 29221 raft_consensus.cc:3060] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.310263 28798 tablet_bootstrap.cc:492] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:29.310402 28798 ts_tablet_manager.cc:1403] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.126s user 0.008s sys 0.009s
I20260618 13:59:29.311209 28798 raft_consensus.cc:359] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.311381 28798 raft_consensus.cc:385] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.311468 28798 raft_consensus.cc:740] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.311685 28798 consensus_queue.cc:260] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.312345 28798 ts_tablet_manager.cc:1434] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 13:59:29.312609 28798 tablet_bootstrap.cc:492] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:29.313021 29221 raft_consensus.cc:2749] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.314033 28799 tablet_bootstrap.cc:492] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:29.314250 28798 tablet_bootstrap.cc:654] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.314278 28799 ts_tablet_manager.cc:1403] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.043s user 0.014s sys 0.003s
I20260618 13:59:29.315305 28799 raft_consensus.cc:359] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.315831 28799 raft_consensus.cc:385] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.315959 28799 raft_consensus.cc:740] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.316334 28799 consensus_queue.cc:260] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.298451 28797 tablet_bootstrap.cc:492] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:29.316916 28797 ts_tablet_manager.cc:1403] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.144s user 0.004s sys 0.017s
I20260618 13:59:29.316923 28799 ts_tablet_manager.cc:1434] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:29.317248 28799 tablet_bootstrap.cc:492] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:29.319975 28797 raft_consensus.cc:359] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.321431 28797 raft_consensus.cc:385] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.321780 28797 raft_consensus.cc:740] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.322239 28797 consensus_queue.cc:260] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.324260 28797 ts_tablet_manager.cc:1434] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260618 13:59:29.325934 28799 tablet_bootstrap.cc:654] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.327597 28797 tablet_bootstrap.cc:492] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:29.329322 28797 tablet_bootstrap.cc:654] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.333271 29402 raft_consensus.cc:493] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.333478 29402 raft_consensus.cc:515] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.334173 29402 leader_election.cc:290] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.334936 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe750fc5d998448caeab0a6c808dfba7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.335166 29362 raft_consensus.cc:493] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.335562 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe750fc5d998448caeab0a6c808dfba7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.335722 28409 raft_consensus.cc:2468] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:29.336774 28618 leader_election.cc:343] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.335500 29362 raft_consensus.cc:515] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.337774 29362 leader_election.cc:290] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.338441 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75824a5d2f7e47838b60e6a2b785c6b2" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.338613 28617 leader_election.cc:304] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
W20260618 13:59:29.339011 28346 leader_election.cc:343] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.342175 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75824a5d2f7e47838b60e6a2b785c6b2" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:29.342886 28346 leader_election.cc:343] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.343050 28346 leader_election.cc:304] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.343456 28797 tablet_bootstrap.cc:492] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:29.343580 29362 raft_consensus.cc:2749] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.343948 29402 raft_consensus.cc:2804] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.344085 29402 raft_consensus.cc:493] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.344237 29402 raft_consensus.cc:3060] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.344453 28799 tablet_bootstrap.cc:492] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:29.344655 28799 ts_tablet_manager.cc:1403] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.004s
I20260618 13:59:29.345705 28799 raft_consensus.cc:359] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.345985 28799 raft_consensus.cc:385] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.346213 28799 raft_consensus.cc:740] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.346634 28799 consensus_queue.cc:260] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.347852 29402 raft_consensus.cc:515] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.348284 28799 ts_tablet_manager.cc:1434] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:29.348749 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe750fc5d998448caeab0a6c808dfba7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.348923 28409 raft_consensus.cc:3060] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.349088 29402 leader_election.cc:290] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.349174 29413 raft_consensus.cc:493] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.344079 28797 ts_tablet_manager.cc:1403] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.004s
I20260618 13:59:29.349370 29413 raft_consensus.cc:515] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.349958 29413 leader_election.cc:290] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.350204 28797 raft_consensus.cc:359] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.350708 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7a97176f72349cda221baaea61d18ad" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.350945 28411 raft_consensus.cc:2468] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:29.350399 28797 raft_consensus.cc:385] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.351469 28617 leader_election.cc:304] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.351853 28797 raft_consensus.cc:740] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.351898 28409 raft_consensus.cc:2468] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.352053 29413 raft_consensus.cc:2804] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.352145 29413 raft_consensus.cc:493] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.352267 29413 raft_consensus.cc:3060] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.352329 28797 consensus_queue.cc:260] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.352957 28617 leader_election.cc:304] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.353376 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe750fc5d998448caeab0a6c808dfba7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:29.353932 29402 raft_consensus.cc:2804] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.354147 29402 raft_consensus.cc:697] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.354493 29402 consensus_queue.cc:237] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.355244 29413 raft_consensus.cc:515] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.355571 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7a97176f72349cda221baaea61d18ad" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.355800 28799 tablet_bootstrap.cc:492] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:29.359099 29413 leader_election.cc:290] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.359774 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7a97176f72349cda221baaea61d18ad" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.360071 28409 raft_consensus.cc:3060] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.362144 28048 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu
/tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.100.60:42083
--webserver_interface=127.27.100.60
--webserver_port=38293
--builtin_ntp_servers=127.27.100.20:36797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.100.62:43001,127.27.100.61:45967,127.27.100.60:42083
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260618 13:59:29.363536 28409 raft_consensus.cc:2468] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
W20260618 13:59:29.356251 28618 leader_election.cc:343] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.355370 28797 ts_tablet_manager.cc:1434] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260618 13:59:29.364174 28797 tablet_bootstrap.cc:492] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:29.364128 28618 leader_election.cc:343] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.365849 28797 tablet_bootstrap.cc:654] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.365900 28617 leader_election.cc:304] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.366629 29299 raft_consensus.cc:2804] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.366771 29299 raft_consensus.cc:697] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.366858 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7a97176f72349cda221baaea61d18ad" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:29.367076 29299 consensus_queue.cc:237] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:29.367378 28618 leader_election.cc:343] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:29.369824 28618 consensus_peers.cc:597] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:29.374303 28799 tablet_bootstrap.cc:654] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.377513 29458 raft_consensus.cc:493] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.377705 29458 raft_consensus.cc:515] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.378381 29458 leader_election.cc:290] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.379688 29413 raft_consensus.cc:493] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.379894 29413 raft_consensus.cc:515] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.382545 29458 raft_consensus.cc:493] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.382735 29458 raft_consensus.cc:515] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.383427 29458 leader_election.cc:290] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:29.380792 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d5e75b4a8247b29d69d07fad5ca5db" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.383882 28410 raft_consensus.cc:2393] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:29.384259 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "092a9aba9c6e48f5be9cf5d55ba93678" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.384582 28409 raft_consensus.cc:2393] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:29.386533 29286 tablet_loader.cc:96] loaded metadata for tablet 55ae29c3b7ed44cfa659130745037aa6 (table default.Table_5d485acde41448d984aa79fa58b1866b [id=726324d365b94ff6bac24030222ca34f])
I20260618 13:59:29.389376 29286 tablet_loader.cc:96] loaded metadata for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_e24e443c392040b0adf1e8c8eb581a10 [id=b97c43b351b946c48f356b1ea1134239])
I20260618 13:59:29.406231 29286 tablet_loader.cc:96] loaded metadata for tablet 5b9f9649587b4c31951204b8bebe7ed8 (table default.Table_f7b8e1561d67469b9a3709475d661de9 [id=caa2a3299e864159b261740eecc0f64d])
I20260618 13:59:29.407932 29286 tablet_loader.cc:96] loaded metadata for tablet 5bbb71449d434d54b007d6f0da0796d7 (table default.Table_571f6cccbfa3470a83b5542f128f2370 [id=9806bb85160042b095abdbcf04cf3748])
I20260618 13:59:29.408123 28799 tablet_bootstrap.cc:492] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:29.408386 28799 ts_tablet_manager.cc:1403] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.053s user 0.008s sys 0.004s
I20260618 13:59:29.409384 28799 raft_consensus.cc:359] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.409704 28799 raft_consensus.cc:385] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.409859 28799 raft_consensus.cc:740] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.410254 28799 consensus_queue.cc:260] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.411000 29286 tablet_loader.cc:96] loaded metadata for tablet 5e25027f423144b088dad955c7282921 (table default.Table_a7ff69763fdb4ff3b26b180bf05907d9 [id=e39e2c5718794a748361d7b453c65404])
I20260618 13:59:29.412345 28799 ts_tablet_manager.cc:1434] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:29.412417 29286 tablet_loader.cc:96] loaded metadata for tablet 650ed7ee8b134bc4b9f95028c54f99be (table default.Table_b5feee88c56b42b6bd1b8ecb81a134bb [id=635bd8a93b22415fab4ee952247c6870])
I20260618 13:59:29.413691 28799 tablet_bootstrap.cc:492] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:29.414237 29286 tablet_loader.cc:96] loaded metadata for tablet 657e55fa53c447babfe91b1582806e6b (table default.Table_b5800aca80594ec1bd3a86ec47dbfc64 [id=2710734517a24ad6baf0d47bf23e9bdf])
I20260618 13:59:29.415930 28799 tablet_bootstrap.cc:654] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.417260 29286 tablet_loader.cc:96] loaded metadata for tablet 69d5e75b4a8247b29d69d07fad5ca5db (table default.Table_76724583c0db40378d257d673a601220 [id=5278d8eb60e4448ebae7afc3edb0db8e])
I20260618 13:59:29.418108 29286 tablet_loader.cc:96] loaded metadata for tablet 6b8f273def3e4b7caa4d624cf87a106c (table default.Table_5159a242b12044fc9a0b3e535ea40050 [id=cfb4ca8c4e884ccc8602be52d0b39100])
I20260618 13:59:29.420532 29286 tablet_loader.cc:96] loaded metadata for tablet 72fd63e8245d4042b091593881ba11b6 (table default.Table_b0e3e2341f764392af42b949d03dd1f6 [id=27b6a19a28f24f2b8a758bbff06312d0])
I20260618 13:59:29.380291 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d5e75b4a8247b29d69d07fad5ca5db" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.381788 29303 raft_consensus.cc:493] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.421113 29303 raft_consensus.cc:515] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.421564 28484 leader_election.cc:304] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.421761 29303 leader_election.cc:290] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:29.384037 28617 consensus_peers.cc:597] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: edc72a06ee8e4420b07a17d2feb3fcc2. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:29.384582 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "092a9aba9c6e48f5be9cf5d55ba93678" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.384423 29402 raft_consensus.cc:493] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.422106 29458 raft_consensus.cc:3060] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.422237 29402 raft_consensus.cc:515] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.422663 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f89c9954d48e407c96c294c26644e25c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.422883 28409 raft_consensus.cc:2468] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:29.422917 29402 leader_election.cc:290] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.424026 28617 leader_election.cc:304] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.424419 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d502d67826ff4d7a8b0f515e5d8b6d77" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.424584 28409 raft_consensus.cc:2468] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:29.425128 28484 leader_election.cc:304] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.425246 29458 raft_consensus.cc:2749] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.425442 29458 raft_consensus.cc:3060] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.425922 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d502d67826ff4d7a8b0f515e5d8b6d77" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.426017 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f89c9954d48e407c96c294c26644e25c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.426561 28617 leader_election.cc:304] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.426923 29303 raft_consensus.cc:2804] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.427037 29303 raft_consensus.cc:493] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.427134 29303 raft_consensus.cc:3060] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.427970 29186 raft_consensus.cc:2804] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.428113 29186 raft_consensus.cc:493] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.428206 29186 raft_consensus.cc:3060] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.430886 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3dbf13544ce408ab97834aec5862390" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.431084 28409 raft_consensus.cc:2468] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:29.431555 28617 leader_election.cc:304] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.432016 29413 leader_election.cc:290] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.432883 29413 raft_consensus.cc:493] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.433058 29413 raft_consensus.cc:515] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.434082 29413 leader_election.cc:290] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.434296 29413 raft_consensus.cc:2804] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.434306 28799 tablet_bootstrap.cc:492] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:29.434479 29413 raft_consensus.cc:493] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.434495 28799 ts_tablet_manager.cc:1403] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.002s
I20260618 13:59:29.434651 29413 raft_consensus.cc:3060] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.434839 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46bfe8cc2078414d84e65e22ed4ef8a2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.435042 28410 raft_consensus.cc:2468] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:29.435523 28483 consensus_peers.cc:597] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 516421cd843b44769efa2b555697c065. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:29.435452 28799 raft_consensus.cc:359] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.435645 28799 raft_consensus.cc:385] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.435755 28799 raft_consensus.cc:740] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.435773 29456 raft_consensus.cc:493] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.435932 29456 raft_consensus.cc:515] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.436035 28799 consensus_queue.cc:260] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.436599 29456 leader_election.cc:290] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:29.436977 29458 raft_consensus.cc:2749] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.437878 28799 ts_tablet_manager.cc:1434] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260618 13:59:29.438295 28483 leader_election.cc:343] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: ac58e939730c4af681c2f08bcfbb6602
I20260618 13:59:29.438820 28617 leader_election.cc:304] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.439286 28799 tablet_bootstrap.cc:492] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:29.439606 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46bfe8cc2078414d84e65e22ed4ef8a2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.439651 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3dbf13544ce408ab97834aec5862390" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:29.440272 28618 leader_election.cc:343] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:29.440645 28618 leader_election.cc:343] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.441036 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac58e939730c4af681c2f08bcfbb6602" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.441267 28799 tablet_bootstrap.cc:654] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.441247 28681 raft_consensus.cc:2468] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:29.441728 29402 raft_consensus.cc:2804] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.441849 29402 raft_consensus.cc:493] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.441962 29402 raft_consensus.cc:3060] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.442853 28484 leader_election.cc:304] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
W20260618 13:59:29.443428 28618 leader_election.cc:343] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:29.443743 28618 leader_election.cc:343] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.445241 29186 raft_consensus.cc:515] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.445238 29303 raft_consensus.cc:515] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.445885 29186 leader_election.cc:290] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.446807 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d502d67826ff4d7a8b0f515e5d8b6d77" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.447047 28410 raft_consensus.cc:3060] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.447455 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f89c9954d48e407c96c294c26644e25c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.447642 28409 raft_consensus.cc:3060] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.450294 28410 raft_consensus.cc:2468] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.451216 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d502d67826ff4d7a8b0f515e5d8b6d77" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:29.451800 28618 leader_election.cc:343] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.451709 28409 raft_consensus.cc:2468] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.452332 28617 leader_election.cc:304] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.452570 29299 raft_consensus.cc:493] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.452744 29299 raft_consensus.cc:515] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.452867 29186 raft_consensus.cc:2804] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.453112 29186 raft_consensus.cc:697] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.453354 29299 leader_election.cc:290] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.453423 29186 consensus_queue.cc:237] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.455054 28617 leader_election.cc:304] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.455403 29301 raft_consensus.cc:2804] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.455598 29301 raft_consensus.cc:697] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.455967 29301 consensus_queue.cc:237] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.457588 29402 raft_consensus.cc:515] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.458084 28799 tablet_bootstrap.cc:492] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:29.458189 29402 leader_election.cc:290] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.458266 28799 ts_tablet_manager.cc:1403] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.001s
I20260618 13:59:29.459115 28799 raft_consensus.cc:359] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.459398 28799 raft_consensus.cc:385] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.459534 28799 raft_consensus.cc:740] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.459710 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46bfe8cc2078414d84e65e22ed4ef8a2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:29.460122 28799 consensus_queue.cc:260] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.460355 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c557afc3abfa4ef188bf248e89f49d87" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:29.460768 28618 leader_election.cc:343] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.460850 29413 raft_consensus.cc:515] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.461963 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46bfe8cc2078414d84e65e22ed4ef8a2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.462244 28410 raft_consensus.cc:3060] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.463653 28798 tablet_bootstrap.cc:492] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:29.464226 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c557afc3abfa4ef188bf248e89f49d87" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.465673 28410 raft_consensus.cc:2468] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.465463 28409 raft_consensus.cc:2468] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:29.467015 28799 ts_tablet_manager.cc:1434] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260618 13:59:29.467917 29362 raft_consensus.cc:493] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.468118 29362 raft_consensus.cc:515] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.468847 29362 leader_election.cc:290] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.471889 28798 ts_tablet_manager.cc:1403] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.159s user 0.008s sys 0.006s
I20260618 13:59:29.472671 28798 raft_consensus.cc:359] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.472872 28798 raft_consensus.cc:385] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.472946 28798 raft_consensus.cc:740] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
W20260618 13:59:29.461547 28618 leader_election.cc:343] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.473361 29286 tablet_loader.cc:96] loaded metadata for tablet 7766014ee9814f01ae7ac3e849ca3d50 (table default.Table_c528008fd94b4fc8b7341f5b23dff76b [id=516d3679e0e84ab28429e59b6895db4e])
I20260618 13:59:29.473711 29413 leader_election.cc:290] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.473846 29286 tablet_loader.cc:96] loaded metadata for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_768d783dc0a7416686369250c3cd855c [id=0321ff97d1674a3891eae8cbba76387a])
I20260618 13:59:29.466988 29458 raft_consensus.cc:2804] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.471083 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b51d0851efb34eaa9e9e4be6b292ef62" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.475150 29458 raft_consensus.cc:493] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.475198 28549 raft_consensus.cc:3060] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.479010 29458 raft_consensus.cc:3060] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.480214 28798 consensus_queue.cc:260] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.481438 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f89c9954d48e407c96c294c26644e25c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:29.482378 29458 raft_consensus.cc:515] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:29.483291 28618 leader_election.cc:343] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.484097 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3dbf13544ce408ab97834aec5862390" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:29.485144 28618 leader_election.cc:343] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.485483 29458 leader_election.cc:290] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:29.486330 29402 consensus_queue.cc:1048] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.486855 29402 consensus_queue.cc:1048] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.487723 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac58e939730c4af681c2f08bcfbb6602" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:29.487933 28683 raft_consensus.cc:3060] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.488266 29286 tablet_loader.cc:96] loaded metadata for tablet 7f83bb8c3c504d5f9affbaa8be3f1508 (table default.Table_a44222ec49e54499bbd3d2c256abc71c [id=3f176321fcf44af2b5a4103634b1908f])
I20260618 13:59:29.488713 29286 tablet_loader.cc:96] loaded metadata for tablet 80e99a85e83c46afa367e59e5ddb7dc7 (table default.Table_334d2fffc20b4a5688939d1067192640 [id=ca499a8d88084635a1bc0c1107ce4b86])
I20260618 13:59:29.488895 28798 ts_tablet_manager.cc:1434] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.017s user 0.001s sys 0.001s
I20260618 13:59:29.489655 29286 tablet_loader.cc:96] loaded metadata for tablet 85b2f34bc17b4a27a91d6cafa9a96c48 (table default.Table_19cfaeff1d174399b77939349481eb5a [id=0cfca3de151f4100b6003fa7b76f0e97])
I20260618 13:59:29.490345 29286 tablet_loader.cc:96] loaded metadata for tablet 897bf6d6bbee4ad89c296325d94a3e53 (table default.Table_35260f90f6814fa3915b4280cff96d1b [id=02ffed3d852845d6845fb5176f036fce])
I20260618 13:59:29.490767 29286 tablet_loader.cc:96] loaded metadata for tablet 8af2bec820b842b4bf6b57482007bd6a (table default.Table_9a10a9aa97574e7c89586d1e4e863171 [id=af1859c644b44c1d831314d27a494f94])
I20260618 13:59:29.491281 28683 raft_consensus.cc:2468] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:29.492125 28484 leader_election.cc:304] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.492532 29458 raft_consensus.cc:2804] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.492707 29458 raft_consensus.cc:697] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:29.463433 29303 leader_election.cc:290] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.466106 28617 leader_election.cc:304] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.493016 29458 consensus_queue.cc:237] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.493530 28798 tablet_bootstrap.cc:492] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:29.493809 28617 leader_election.cc:304] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.470271 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b51d0851efb34eaa9e9e4be6b292ef62" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.496230 29286 tablet_loader.cc:96] loaded metadata for tablet 8ed88c142d7d40cda6392c06bc0486ca (table default.Table_ddb8f9243a5d47d1be63b3a371ca17ac [id=225463c2467047edae05b73a3132638f])
I20260618 13:59:29.496773 29286 tablet_loader.cc:96] loaded metadata for tablet 8f7b66652f464893ad812d27e883a9a4 (table default.Table_697444611a2643bfb1fe30fc545c47f3 [id=4aece4c39f65473494e823b598e81bd5])
I20260618 13:59:29.498983 28798 tablet_bootstrap.cc:654] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.500761 28797 tablet_bootstrap.cc:492] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:29.500983 28797 ts_tablet_manager.cc:1403] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.137s user 0.004s sys 0.001s
I20260618 13:59:29.502027 28797 raft_consensus.cc:359] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.502272 28797 raft_consensus.cc:385] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.502405 28797 raft_consensus.cc:740] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.502743 28797 consensus_queue.cc:260] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.503414 28797 ts_tablet_manager.cc:1434] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 13:59:29.507047 29362 raft_consensus.cc:493] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.507308 29286 tablet_loader.cc:96] loaded metadata for tablet 8fea86ca38a741238a59053151087f69 (table default.Table_7ae70108b1714e5ca374cc653bed7b33 [id=0cf9d7859b7d4d95bf769070c43003d3])
I20260618 13:59:29.508168 29362 raft_consensus.cc:515] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.508275 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3dbf13544ce408ab97834aec5862390" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.508888 29286 tablet_loader.cc:96] loaded metadata for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f])
I20260618 13:59:29.509264 28409 raft_consensus.cc:3060] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.509634 29286 tablet_loader.cc:96] loaded metadata for tablet 91d014b46a0f448b929cbc9287ead0a7 (table default.Table_62bf5ce52fb14bd089678663dd009a91 [id=713af0f85a434ed0bfb791c48f75dc9d])
W20260618 13:59:29.509970 28346 leader_election.cc:343] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.510107 29286 tablet_loader.cc:96] loaded metadata for tablet 92ddbdfdfff547588a03f453e413af8f (table default.Table_93b6c79a288040ceae924af14f939c35 [id=61ce711b673b44189a507d1abf26bd75])
W20260618 13:59:29.510272 28483 leader_election.cc:343] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: ac58e939730c4af681c2f08bcfbb6602
I20260618 13:59:29.512223 29303 raft_consensus.cc:2804] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
W20260618 13:59:29.512710 28346 leader_election.cc:343] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.512722 29303 raft_consensus.cc:697] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.512861 28346 leader_election.cc:304] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.513017 29303 consensus_queue.cc:237] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.513659 29286 tablet_loader.cc:96] loaded metadata for tablet 9559c5f2c9e94e858093dd2f0865c865 (table default.Table_38a3b325a9514a3d829c6ba288f9673a [id=8f0879e0c0b74edfa8ae99d11cda07a1])
I20260618 13:59:29.514108 29286 tablet_loader.cc:96] loaded metadata for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b])
I20260618 13:59:29.514264 29186 raft_consensus.cc:2804] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.514607 29186 raft_consensus.cc:493] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.514747 29186 raft_consensus.cc:3060] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.515288 29286 tablet_loader.cc:96] loaded metadata for tablet 962bf304a285412fb2a529db5b994e9e (table default.Table_0fbace1c1aa746a6a2a57c878631218b [id=18b5971926ed4fff9048003a51d3a4fa])
I20260618 13:59:29.515702 29286 tablet_loader.cc:96] loaded metadata for tablet 97a9f949999446f09359629fe483bf67 (table default.Table_9424eec16a284672a0fe7ab3d0852b40 [id=070ea48af78346aa8ff43f2bb369730c])
I20260618 13:59:29.516010 29183 raft_consensus.cc:2749] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.517096 29301 consensus_queue.cc:1048] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.517369 29183 consensus_queue.cc:1048] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260618 13:59:29.519876 29286 tablet_loader.cc:96] loaded metadata for tablet 98446f40214c4ac984fcfe98310005c5 (table default.Table_83d021ed148243ffb453cb5e2484b7b4 [id=b3cae943ecdf410ca1cf711c5ddfe14a])
I20260618 13:59:29.520445 29286 tablet_loader.cc:96] loaded metadata for tablet 98b42d23832545ae81a7cdff19e4689f (table default.Table_c99584ecc88948a097b39c22d68b3938 [id=7c39b3c432e944bca05f7b2070a79ff9])
I20260618 13:59:29.520990 29286 tablet_loader.cc:96] loaded metadata for tablet 9a0691de253b4b008e5d995fd6b876c2 (table default.Table_ab57b782a92747e78e2bfe28480aa0ec [id=28d43fd5660f459a9acfab2fc0dfbdb1])
I20260618 13:59:29.521546 29286 tablet_loader.cc:96] loaded metadata for tablet 9ab1ddf79b094ba2aeee9225c0b332c3 (table default.Table_8e7925d0e7eb421685f09b8c53667351 [id=bbe67818f204445c89ede107910a3bdd])
I20260618 13:59:29.522069 29286 tablet_loader.cc:96] loaded metadata for tablet 9b75ece59d924e3db49e002bf0faa89c (table default.Table_031e1a03030b45c99c1270d612008f91 [id=387d3ff1762749efad926181fe4c746a])
I20260618 13:59:29.522619 29286 tablet_loader.cc:96] loaded metadata for tablet 9c17e76ae9b44207998a43e3576908c4 (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e])
I20260618 13:59:29.523020 29286 tablet_loader.cc:96] loaded metadata for tablet 9d8f124ba79b46e098303b7b2705844d (table default.Table_6f1f355b18a34ac09304992c96d40765 [id=7bb817b09d154696977f3d68d9461d4e])
I20260618 13:59:29.523600 29286 tablet_loader.cc:96] loaded metadata for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_efb2b8e451084b7ba0de07e0a6e538c2 [id=333c0fd3837b4d89a9197eb6a826c311])
I20260618 13:59:29.524257 28797 tablet_bootstrap.cc:492] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:29.525054 29286 tablet_loader.cc:96] loaded metadata for tablet 9fa1a27fc9304e18b96cad9f923779f2 (table default.Table_aeeec1571f064ab1816cbf26635190fd [id=e408f407250f4c0eb56e1abe07fb6b02])
I20260618 13:59:29.526275 28409 raft_consensus.cc:2468] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.526322 29286 tablet_loader.cc:96] loaded metadata for tablet a10e83d26b704201b95e7c0c44a1a4f5 (table default.Table_663ed02889aa45938d7d1316d4d19c06 [id=c79cfc8300a94a18873df52a1d83ada3])
I20260618 13:59:29.527204 29286 tablet_loader.cc:96] loaded metadata for tablet a1a06078974e4bfbbc4b949802a39bf1 (table default.Table_f958c4ce58c34bfda98454664e202af2 [id=7b747ef3b2494ca78c499c5bb75648d4])
I20260618 13:59:29.527861 29286 tablet_loader.cc:96] loaded metadata for tablet a3c74d3ca66f42ad83ef54d9459b99fd (table default.Table_dabd3b5bd3ff4784b839d502baec0c6e [id=79633bfcdf584058b406f418caa25aea])
I20260618 13:59:29.529872 29286 tablet_loader.cc:96] loaded metadata for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b])
I20260618 13:59:29.530462 29286 tablet_loader.cc:96] loaded metadata for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_9cf9f72ae27b44bc9aa43303a7103214 [id=f50e59ae00344758833392841b7f4985])
I20260618 13:59:29.530532 28617 leader_election.cc:304] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.530831 29401 raft_consensus.cc:2804] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.531476 29286 tablet_loader.cc:96] loaded metadata for tablet ad045effb32948b5b8e9be5bb7996577 (table default.Table_7910db46cf394cdebe608e7efb3388cc [id=421bf4faf8174ccd88dc15799671c74d])
I20260618 13:59:29.531775 29401 raft_consensus.cc:697] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.532060 29401 consensus_queue.cc:237] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.533200 29286 tablet_loader.cc:96] loaded metadata for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522])
W20260618 13:59:29.534653 28617 consensus_peers.cc:597] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c84189a4b1194c87bbdf81f6e0ed56e4. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:29.539521 28799 tablet_bootstrap.cc:492] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:29.541333 29362 leader_election.cc:290] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.541596 28799 tablet_bootstrap.cc:654] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.545384 28798 tablet_bootstrap.cc:492] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:29.545542 28798 ts_tablet_manager.cc:1403] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.052s user 0.001s sys 0.002s
I20260618 13:59:29.546379 28798 raft_consensus.cc:359] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.546589 28798 raft_consensus.cc:385] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.546705 28798 raft_consensus.cc:740] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.552362 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02b5cc08ca0c4e719e0414b345211ba5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.552340 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02b5cc08ca0c4e719e0414b345211ba5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:29.553979 28346 leader_election.cc:343] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:29.554277 28346 leader_election.cc:343] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.554479 28346 leader_election.cc:304] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.554857 29183 raft_consensus.cc:2749] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.550933 28798 consensus_queue.cc:260] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.555109 29183 consensus_queue.cc:1048] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.556169 29458 raft_consensus.cc:493] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.556366 29458 raft_consensus.cc:515] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.557039 29458 leader_election.cc:290] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.557896 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fddea8fba4da40d7a5780e6750caf5e9" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.558123 28409 raft_consensus.cc:2393] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:29.568579 29286 tablet_loader.cc:96] loaded metadata for tablet b509f940a02545019c526e45e8372c97 (table default.Table_12d6cc95071949c6a7a297cd08b5f898 [id=840c933184144aebb3659d939f679f00])
I20260618 13:59:29.569358 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fddea8fba4da40d7a5780e6750caf5e9" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.570238 29286 tablet_loader.cc:96] loaded metadata for tablet b95bdb0e36584d92b9c28de257553827 (table default.Table_259b4dd1855b42a8824f00306cb5223a [id=15c34c0db3b54b27ac29f13880f49f0d])
I20260618 13:59:29.570633 28484 leader_election.cc:304] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.571399 29456 raft_consensus.cc:3060] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.572338 29286 tablet_loader.cc:96] loaded metadata for tablet b968a8d4014341ca9c5964263721ac79 (table default.Table_b8741a7fd29f4e78a948b1c419595a85 [id=6f4d908593ae40f09e2c94cef218e1fd])
I20260618 13:59:29.573011 29286 tablet_loader.cc:96] loaded metadata for tablet ba3d38d32c7c4721bad35f0da7a9ebe5 (table default.Table_40fdb9a7e5ad4ff2bf879ec4e02a4b8d [id=c1e1069f4a1640e2a3f86b302f1dcc7a])
I20260618 13:59:29.573431 29286 tablet_loader.cc:96] loaded metadata for tablet ba53ad1988064049a61fcca3a85f8779 (table default.Table_2a3bcaddd3034d67a5ea7c78f0ea9e7c [id=6c30b030a60b47e2be48dfe6e9cfdfdb])
I20260618 13:59:29.573822 29286 tablet_loader.cc:96] loaded metadata for tablet bab50e1f17a74eee8cad942525002996 (table default.Table_8f22977cfa824da99508b4063a05f8e6 [id=b527fd0afd76403f94cb5fe9f4c95094])
I20260618 13:59:29.574667 29456 raft_consensus.cc:2749] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.555619 28798 ts_tablet_manager.cc:1434] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.010s user 0.000s sys 0.002s
I20260618 13:59:29.575261 29286 tablet_loader.cc:96] loaded metadata for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230])
I20260618 13:59:29.576653 29286 tablet_loader.cc:96] loaded metadata for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_7cd820b0428a41ada19a2f14560a67e4 [id=2c73a9d87a914f03b09ef422d49f7a95])
I20260618 13:59:29.577155 29286 tablet_loader.cc:96] loaded metadata for tablet c01c8463055e4526b3c75190066f30b6 (table default.Table_7e5d58ad3e8043919b66e8890819535c [id=96aff34b68c1494f9228a11eb38de4c7])
I20260618 13:59:29.577569 29286 tablet_loader.cc:96] loaded metadata for tablet c0a42aa779ae452e9420d730bf8d7d55 (table default.Table_fac3fdd9b466497683bc00300a85fed1 [id=a4ac38531af34b4681abba96b42ffb7f])
I20260618 13:59:29.577704 29301 consensus_queue.cc:1048] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.578296 29286 tablet_loader.cc:96] loaded metadata for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_dc44ae6b83d844b6a16ce10ba6dce0ad [id=2e7a199c805d4453b2137b5fa69d0b4e])
I20260618 13:59:29.579052 29286 tablet_loader.cc:96] loaded metadata for tablet c4cfca9e76e641a7bc0b58b9ee240f36 (table default.Table_0e9f574416d84a0f826fa9a1fcc56210 [id=098fa24bb71e4fe1aecc33aea1275ef0])
I20260618 13:59:29.580219 29286 tablet_loader.cc:96] loaded metadata for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a])
I20260618 13:59:29.581509 29286 tablet_loader.cc:96] loaded metadata for tablet c871e7c8f843486da7ba40145c0fed19 (table default.Table_c6e2f443374a47d2a787049499a0b168 [id=e875c5c4a0004e8287426403b0b9c626])
I20260618 13:59:29.582000 29286 tablet_loader.cc:96] loaded metadata for tablet c881dd3cb66344969310956ad6e83cc0 (table default.Table_d31dc25d118d4cd2bb4dd8f034752827 [id=b1d85ea3aa7047d285ab7c86da8213fd])
I20260618 13:59:29.582466 29286 tablet_loader.cc:96] loaded metadata for tablet ca2ee746c5024cfc9b31c470f9616e7d (table default.Table_72e5809d8cce4f3bbe8227efcb8e2b2c [id=dcd71daa60ee430d9ca46a1463d9a1bf])
I20260618 13:59:29.583176 29286 tablet_loader.cc:96] loaded metadata for tablet cd06bb7cffc34412b420d70435fce373 (table default.Table_74fc7c21dc664ff8bfe0d25aee399bdb [id=d224150fd3a340fc96e75c0b96146b11])
I20260618 13:59:29.583647 29286 tablet_loader.cc:96] loaded metadata for tablet cea4bc13eabc4dd98b4f96723498014c (table default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c])
I20260618 13:59:29.584093 29286 tablet_loader.cc:96] loaded metadata for tablet d0bcff62290c43c9be7cc496fbff6142 (table default.Table_bfcd14e3d49e4de09270ed85ec9cde4c [id=aa0452c5d31844c4b99cd207c0163445])
W20260618 13:59:29.584148 28617 consensus_peers.cc:597] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2188ebf1727240cd886ba7662f8fcf43. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:29.584558 29286 tablet_loader.cc:96] loaded metadata for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_7fa3da42543d4978aac5ac383ed10056 [id=9102540068234d1993fa164035dabc6e])
I20260618 13:59:29.585115 29186 raft_consensus.cc:515] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.585150 29299 consensus_queue.cc:1048] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.047s
I20260618 13:59:29.585282 29286 tablet_loader.cc:96] loaded metadata for tablet d2d54a901e9f4d5c89700979ce0618bf (table default.Table_0ec7ef7f4c6e48d5b196e45f78a4fbf2 [id=3b33c0ca00d248298402d6bcec792be4])
I20260618 13:59:29.585323 29402 consensus_queue.cc:1048] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260618 13:59:29.585661 29286 tablet_loader.cc:96] loaded metadata for tablet d3dbf13544ce408ab97834aec5862390 (table default.Table_bf5ce6c6e0b64002994fb45e76ab30eb [id=a0e6904d88704db39c194c34ac9a7faf])
I20260618 13:59:29.586026 29286 tablet_loader.cc:96] loaded metadata for tablet d4539f767c054e9e8ebc1f3f56108e7e (table default.Table_d838306823ff4e22a68efe0a8cb751ff [id=bbfd0a6a47db4b1b8c534405b1068d5e])
I20260618 13:59:29.586475 29286 tablet_loader.cc:96] loaded metadata for tablet d464dd402d2044469e364ed37fdfb39b (table default.Table_e8ae99dfd8e84d7bb2723eb5fe129c2f [id=6d8f3597814241f3bc8c920d3a59008e])
I20260618 13:59:29.586946 29286 tablet_loader.cc:96] loaded metadata for tablet d4667fe48f0f4eff9c6c8baabdab58d4 (table default.Table_0a85d79af65f48a3803bfd1248593a11 [id=2f4d62be772a421ab777fd23be58b719])
I20260618 13:59:29.587535 29400 consensus_queue.cc:1048] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.589990 29456 consensus_queue.cc:1048] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.600193 29400 consensus_queue.cc:1048] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.602092 29286 tablet_loader.cc:96] loaded metadata for tablet dc36fbea853a4d4e9f61909200239aed (table default.Table_dbc6162c06f74e71b792bd1f7b2388fe [id=49576dabdecf41aca1b15ff3fc0c113c])
I20260618 13:59:29.606802 29400 consensus_queue.cc:1048] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.608300 29400 raft_consensus.cc:493] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.608522 29400 raft_consensus.cc:515] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.609607 29400 leader_election.cc:290] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.610277 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b75ece59d924e3db49e002bf0faa89c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.610546 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b75ece59d924e3db49e002bf0faa89c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.610831 28407 raft_consensus.cc:2468] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:29.611267 29286 tablet_loader.cc:96] loaded metadata for tablet e5f7eaa7d218440ca2c35340c5c87814 (table default.Table_9a78338c3e414b0abea1281082202f57 [id=5697846ccd9c4c659755a0f9b8a66b72])
I20260618 13:59:29.611500 28547 raft_consensus.cc:3060] T 9a0691de253b4b008e5d995fd6b876c2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.612270 29286 tablet_loader.cc:96] loaded metadata for tablet e86aeb21427e4859a5266f4451469b25 (table default.Table_7e0256d5218f469c9fef22ba76bb4e64 [id=65db0daf3f084f09bfd0bba3011741d9])
I20260618 13:59:29.613099 28617 leader_election.cc:304] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.613267 29286 tablet_loader.cc:96] loaded metadata for tablet ece7909d056c4ba8a5dca6423f0c1781 (table default.Table_26e7d8c4f91144268a2a43e83992667d [id=803d5220ce1a4dcfaf27d93c574be2ac])
I20260618 13:59:29.613517 29303 raft_consensus.cc:2804] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.613677 29286 tablet_loader.cc:96] loaded metadata for tablet ed648dcf882340388155068d7a5a7869 (table default.Table_8c9c9a6eea174bd68d8fb9d9ebaaf603 [id=185e7792e79644a2aced2b949fca635b])
I20260618 13:59:29.613742 29303 raft_consensus.cc:493] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.613965 29303 raft_consensus.cc:3060] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:29.614846 28618 leader_election.cc:343] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:29.617972 28483 consensus_peers.cc:597] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4a01dfb4d59549399c94640f16045555. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:29.618973 29286 tablet_loader.cc:96] loaded metadata for tablet eebebb79db644535b54a48e4dee89d59 (table default.Table_84c59c9ab3f54fb4930771088e194560 [id=ee8aa366b2d1461099cb328367424ebd])
I20260618 13:59:29.618934 29299 consensus_queue.cc:1048] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.619771 29286 tablet_loader.cc:96] loaded metadata for tablet f1663f9d92e54f1caf3c7137412e495c (table default.Table_0a074f8e7c5443959918f5effc28e3d7 [id=e78c685d531d4d1b831e33f2b059b90c])
I20260618 13:59:29.620236 29286 tablet_loader.cc:96] loaded metadata for tablet f278d0acbb0c4a0ca61640bc142e6335 (table default.Table_8783a8d8c526428281436b62b2581a33 [id=0d83fc68255c458587390c58b12e00d7])
I20260618 13:59:29.620607 28798 tablet_bootstrap.cc:492] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:29.620671 29286 tablet_loader.cc:96] loaded metadata for tablet f2b21dcf74fa4264947ca43cc0026e76 (table default.Table_c35470a550534fa2ad072f932e981df5 [id=2797c7fefad74c1e82ab31fb9ad4db70])
I20260618 13:59:29.623761 29303 raft_consensus.cc:515] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.625209 28797 tablet_bootstrap.cc:654] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.627462 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b75ece59d924e3db49e002bf0faa89c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:29.628098 29186 leader_election.cc:290] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.628182 29303 leader_election.cc:290] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.628436 29186 consensus_queue.cc:1048] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.628808 29286 tablet_loader.cc:96] loaded metadata for tablet f3ba9186b3ff47d5a2b158fef619686e (table default.Table_3ef2ff183a6044e2a9e069a4eac133b8 [id=60e91b2138bf41a5a8dee65c1874678a])
I20260618 13:59:29.629159 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b75ece59d924e3db49e002bf0faa89c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.629385 28407 raft_consensus.cc:3060] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.629654 29286 tablet_loader.cc:96] loaded metadata for tablet f45556a59c424ca2bc30956566374c89 (table default.Table_9c526031302044508f50fead286b4465 [id=408b312e0fad42ae8fb0ab84abd85c0b])
W20260618 13:59:29.629783 28618 leader_election.cc:343] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.630101 29286 tablet_loader.cc:96] loaded metadata for tablet f54910015987484e95b295352d859315 (table default.Table_e97ff5cd68d24da7a9a803e10b35dbe6 [id=2c1b442aeecf44ee8b6140cacdb37627])
I20260618 13:59:29.630532 29286 tablet_loader.cc:96] loaded metadata for tablet f695a7e6a925470787a708c365e6b86b (table default.Table_9763de0aea1d44d785311a303abb1d27 [id=34c58ef63c21476685cbc59915bbfd0a])
I20260618 13:59:29.631678 29286 tablet_loader.cc:96] loaded metadata for tablet f9f7359c4a144ca89fe3e51d1365a425 (table default.Table_7d317fa28e3142a19d15c8360151c357 [id=bba3ab8821ca48a5aa1104094d78892a])
I20260618 13:59:29.631879 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c557afc3abfa4ef188bf248e89f49d87" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:29.632051 29286 tablet_loader.cc:96] loaded metadata for tablet fa24afe9db46408cace50f642f879f4b (table default.Table_4108c7ea79f5426395746346c44df521 [id=5a04afb74af64b17914f1587d7ddc542])
I20260618 13:59:29.632438 29286 tablet_loader.cc:96] loaded metadata for tablet fc1968e5ce2e42a5b836c90b0904f158 (table default.Table_dfe94c1f858644c98dba7db12d019477 [id=ea67f187738d41c993153f788e016e73])
W20260618 13:59:29.632696 28618 leader_election.cc:343] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.633131 28407 raft_consensus.cc:2468] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.633801 29286 tablet_loader.cc:96] loaded metadata for tablet fc295a8e81b64859915a6335f8fb8d8e (table default.Table_21633c55ca8c4614a2a0b241a0b89ee1 [id=411f4d0d27f94aa9842287ba693b2a01])
I20260618 13:59:29.634557 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c557afc3abfa4ef188bf248e89f49d87" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.634845 28407 raft_consensus.cc:3060] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.635843 29286 tablet_loader.cc:96] loaded metadata for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e])
I20260618 13:59:29.637023 29286 catalog_manager.cc:1519] Initializing Kudu cluster ID...
I20260618 13:59:29.638052 29048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 501 } } ignore_live_leader: false dest_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" is_pre_election: true
I20260618 13:59:29.638353 28407 raft_consensus.cc:2468] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
W20260618 13:59:29.638397 29048 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33136 (request call id 4) took 542 ms (client timeout 665 ms). Trace:
I20260618 13:59:29.639075 28617 leader_election.cc:304] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
W20260618 13:59:29.638571 29048 rpcz_store.cc:269] 0618 13:59:29.095420 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:29.095659 (+ 239us) service_pool.cc:224] Handling call
0618 13:59:29.638328 (+542669us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260618 13:59:29.640007 29413 raft_consensus.cc:2804] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.640033 28074 leader_election.cc:304] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e4d4f4566be42dfbafb7c0e40fb17ff; no voters: 7637285570574a6dbcef1de6f84ecac0, a950ab7cd66b4cb0b29592aa692c2dd0
I20260618 13:59:29.639372 29400 raft_consensus.cc:493] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.640635 29400 raft_consensus.cc:515] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.641691 29299 consensus_queue.cc:1048] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.642025 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8cf927bed9437086efd8e0f21ba26a" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:29.642858 28618 leader_election.cc:343] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.643038 29286 catalog_manager.cc:1294] Loaded cluster ID: 7094aaeda1bc41d1ac58a313dfa3bf98
I20260618 13:59:29.643144 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8cf927bed9437086efd8e0f21ba26a" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.643371 28411 raft_consensus.cc:2468] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:29.643529 29508 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260618 13:59:29.643153 29286 catalog_manager.cc:1530] Initializing Kudu internal certificate authority...
I20260618 13:59:29.643658 29413 raft_consensus.cc:697] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.643951 29413 consensus_queue.cc:237] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.643903 28617 leader_election.cc:304] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.644994 29303 raft_consensus.cc:2804] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.645223 29303 raft_consensus.cc:697] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.645275 28617 leader_election.cc:304] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.645583 29303 consensus_queue.cc:237] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.645704 29402 raft_consensus.cc:2804] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.645838 29402 raft_consensus.cc:493] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.645931 29402 raft_consensus.cc:3060] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.641755 29400 leader_election.cc:290] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.649003 29402 raft_consensus.cc:515] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.649607 29402 leader_election.cc:290] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.650945 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8cf927bed9437086efd8e0f21ba26a" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.651177 28411 raft_consensus.cc:3060] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.652402 29286 catalog_manager.cc:1539] Loading token signing keys...
I20260618 13:59:29.654060 28411 raft_consensus.cc:2468] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.654799 29489 raft_consensus.cc:493] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.655011 29489 raft_consensus.cc:515] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.655934 29489 leader_election.cc:290] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.656821 28617 leader_election.cc:304] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.656785 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba3d38d32c7c4721bad35f0da7a9ebe5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.657148 29303 raft_consensus.cc:2804] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.657433 29303 raft_consensus.cc:697] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.657461 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba3d38d32c7c4721bad35f0da7a9ebe5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:29.657656 28346 leader_election.cc:343] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:29.657903 28346 leader_election.cc:343] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.658113 29286 catalog_manager.cc:6080] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Loaded TSK: 0
I20260618 13:59:29.658110 28346 leader_election.cc:304] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.658631 29303 consensus_queue.cc:237] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.659444 29286 catalog_manager.cc:1549] Initializing in-progress tserver states...
I20260618 13:59:29.659860 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8cf927bed9437086efd8e0f21ba26a" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:29.659940 29489 raft_consensus.cc:2749] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:29.660509 28618 leader_election.cc:343] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.663686 29402 consensus_queue.cc:1048] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.666383 28798 tablet_bootstrap.cc:654] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.678716 29186 raft_consensus.cc:493] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.679107 29186 raft_consensus.cc:515] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.679998 29186 leader_election.cc:290] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.681038 28545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4144b1b5ea4e968672616bc22bead8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.683210 28797 tablet_bootstrap.cc:492] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
W20260618 13:59:29.683269 28618 leader_election.cc:343] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.687425 28799 tablet_bootstrap.cc:492] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:29.687691 28799 ts_tablet_manager.cc:1403] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.216s user 0.003s sys 0.003s
I20260618 13:59:29.690904 28797 ts_tablet_manager.cc:1403] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.180s user 0.004s sys 0.001s
I20260618 13:59:29.691843 28797 raft_consensus.cc:359] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.692436 28799 raft_consensus.cc:359] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.701309 29489 raft_consensus.cc:493] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.701463 29489 raft_consensus.cc:515] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.702096 29489 leader_election.cc:290] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.700488 29026 catalog_manager.cc:5696] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } }
I20260618 13:59:29.704062 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9c17e76ae9b44207998a43e3576908c4 (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]). Expected version 2 got 0
I20260618 13:59:29.704531 29026 catalog_manager.cc:5696] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } }
I20260618 13:59:29.704948 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 69d5e75b4a8247b29d69d07fad5ca5db (table default.Table_76724583c0db40378d257d673a601220 [id=5278d8eb60e4448ebae7afc3edb0db8e]). Expected version 1 got 0
I20260618 13:59:29.705238 29026 catalog_manager.cc:5696] T 9a0691de253b4b008e5d995fd6b876c2 P 97859748af2c41879ba2a668dda999d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } }
I20260618 13:59:29.693094 29402 raft_consensus.cc:493] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.695602 29303 consensus_queue.cc:1048] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.705657 29402 raft_consensus.cc:515] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.706218 29402 leader_election.cc:290] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.709136 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9bd8577f4f4a9593a3ffeab582b02e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.709537 28797 raft_consensus.cc:385] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.709685 28797 raft_consensus.cc:740] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.709867 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e95c88e4b0a046a3ad69ee49af47c7c3" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.710042 28797 consensus_queue.cc:260] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:29.711256 28618 leader_election.cc:343] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.711979 28797 ts_tablet_manager.cc:1434] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.021s user 0.002s sys 0.000s
I20260618 13:59:29.713459 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9bd8577f4f4a9593a3ffeab582b02e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.714162 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4144b1b5ea4e968672616bc22bead8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
W20260618 13:59:29.715268 28618 consensus_peers.cc:597] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:29.717124 28797 tablet_bootstrap.cc:492] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:29.718089 28346 leader_election.cc:343] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:29.718452 28346 leader_election.cc:343] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.718631 28346 leader_election.cc:304] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.718909 29362 raft_consensus.cc:2749] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.719637 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e95c88e4b0a046a3ad69ee49af47c7c3" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.719705 28407 raft_consensus.cc:2468] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:29.719868 28409 raft_consensus.cc:2468] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:29.720259 28617 leader_election.cc:304] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.720786 28617 leader_election.cc:304] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.721047 29402 consensus_queue.cc:1048] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.721386 29402 raft_consensus.cc:2804] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.721477 29402 raft_consensus.cc:493] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.721553 29402 raft_consensus.cc:3060] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.722028 29186 raft_consensus.cc:2804] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.722118 29186 raft_consensus.cc:493] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.722225 29186 raft_consensus.cc:3060] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.722287 29456 consensus_queue.cc:1048] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.722550 28797 tablet_bootstrap.cc:654] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.725119 29186 raft_consensus.cc:515] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.725689 29186 leader_election.cc:290] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.725843 29186 raft_consensus.cc:493] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.726016 29402 raft_consensus.cc:515] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.726915 29402 leader_election.cc:290] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.727968 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e95c88e4b0a046a3ad69ee49af47c7c3" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.728205 28407 raft_consensus.cc:3060] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.729743 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4144b1b5ea4e968672616bc22bead8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.729897 28411 raft_consensus.cc:3060] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.733302 29024 master_service.cc:960] ReplaceTablet: received request to replace tablet 5e25027f423144b088dad955c7282921 from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:29.734738 29020 master_service.cc:960] ReplaceTablet: received request to replace tablet d4667fe48f0f4eff9c6c8baabdab58d4 from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:29.734895 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e95c88e4b0a046a3ad69ee49af47c7c3" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:29.735360 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d4144b1b5ea4e968672616bc22bead8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:29.735905 29402 consensus_queue.cc:1048] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.736719 28797 tablet_bootstrap.cc:492] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:29.736881 28797 ts_tablet_manager.cc:1403] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.004s
W20260618 13:59:29.736928 28618 leader_election.cc:343] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:29.737160 28618 leader_election.cc:343] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.737329 29025 master_service.cc:960] ReplaceTablet: received request to replace tablet 015494374645469785f1f978eaf0e9a5 from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:29.738699 28799 raft_consensus.cc:385] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.725991 29186 raft_consensus.cc:515] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.742120 28798 tablet_bootstrap.cc:492] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:29.744046 28799 raft_consensus.cc:740] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.744326 28799 consensus_queue.cc:260] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.745286 28799 ts_tablet_manager.cc:1434] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.057s user 0.001s sys 0.001s
I20260618 13:59:29.749622 28797 raft_consensus.cc:359] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.749959 28797 raft_consensus.cc:385] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.750823 28797 raft_consensus.cc:740] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
W20260618 13:59:29.750947 28346 consensus_peers.cc:597] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:29.751215 28797 consensus_queue.cc:260] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.753007 28411 raft_consensus.cc:2468] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.753461 29186 leader_election.cc:290] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.753989 29362 consensus_queue.cc:1048] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.758251 28799 tablet_bootstrap.cc:492] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:29.761370 28407 raft_consensus.cc:2468] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.763015 28799 tablet_bootstrap.cc:654] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.769730 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e86aeb21427e4859a5266f4451469b25" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.769987 28410 raft_consensus.cc:2468] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:29.772238 28617 leader_election.cc:304] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.778551 28797 ts_tablet_manager.cc:1434] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.034s user 0.000s sys 0.003s
I20260618 13:59:29.778631 29186 raft_consensus.cc:2804] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.778954 28797 tablet_bootstrap.cc:492] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:29.778581 28617 leader_election.cc:304] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.780443 29303 raft_consensus.cc:2804] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.780622 29303 raft_consensus.cc:697] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.780937 29303 consensus_queue.cc:237] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.780474 28617 leader_election.cc:304] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:29.782048 29402 raft_consensus.cc:2804] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.782217 29402 raft_consensus.cc:493] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.782344 29402 raft_consensus.cc:3060] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.783236 28797 tablet_bootstrap.cc:654] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.785646 29402 raft_consensus.cc:515] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.735750 29019 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_b703123b497f445b89066c7ec657f91c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:29.757822 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e86aeb21427e4859a5266f4451469b25" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.789422 29402 leader_election.cc:290] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:29.789759 28618 leader_election.cc:343] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.789893 29186 raft_consensus.cc:697] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.742297 28798 ts_tablet_manager.cc:1403] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.167s user 0.004s sys 0.001s
I20260618 13:59:29.790210 29186 consensus_queue.cc:237] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.790685 29522 raft_consensus.cc:493] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260618 13:59:29.790905 29019 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b703123b497f445b89066c7ec657f91c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:29.790855 29522 raft_consensus.cc:515] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.791636 29522 leader_election.cc:290] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:29.792263 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1baf86497d442dbbbddbc3498bebbee" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:29.736042 29018 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_819e5138e3c7415a80631f3cbb6f8b81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:29.793164 29018 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_819e5138e3c7415a80631f3cbb6f8b81 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:29.793273 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1baf86497d442dbbbddbc3498bebbee" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:29.793845 28346 leader_election.cc:343] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:29.794056 28346 leader_election.cc:343] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.794167 28346 leader_election.cc:304] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.736397 29017 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_9424eec16a284672a0fe7ab3d0852b40" } modify_external_catalogs: true
I20260618 13:59:29.794446 29017 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_9424eec16a284672a0fe7ab3d0852b40" } modify_external_catalogs: true
I20260618 13:59:29.794538 29522 raft_consensus.cc:2749] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.736651 29011 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_76b9ce534bf440919c76e1c8005ecc57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:29.796677 29011 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_76b9ce534bf440919c76e1c8005ecc57 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:29.736989 29007 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_7cd820b0428a41ada19a2f14560a67e4" } new_table_name: "default.Table_f665d2e34fc24b29b021a45c8b57a700" modify_external_catalogs: true
I20260618 13:59:29.801175 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e86aeb21427e4859a5266f4451469b25" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:29.802898 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e86aeb21427e4859a5266f4451469b25" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.803155 28407 raft_consensus.cc:3060] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:29.806089 28617 consensus_peers.cc:597] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4c2b924a096b479cad31c29bf7c99172. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:29.807468 29413 consensus_queue.cc:1048] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:29.808830 28618 leader_election.cc:343] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.806432 28798 raft_consensus.cc:359] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.807508 29456 raft_consensus.cc:493] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.809376 28407 raft_consensus.cc:2468] T e86aeb21427e4859a5266f4451469b25 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.809372 29456 raft_consensus.cc:515] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.810076 29456 leader_election.cc:290] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.811044 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd2385e6752d49429422fc854539e9c6" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.811928 28798 raft_consensus.cc:385] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.812132 28798 raft_consensus.cc:740] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.812646 28617 leader_election.cc:304] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.812853 28799 tablet_bootstrap.cc:492] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:29.813767 28799 ts_tablet_manager.cc:1403] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.056s user 0.006s sys 0.006s
I20260618 13:59:29.814435 28798 consensus_queue.cc:260] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.814791 29413 raft_consensus.cc:2804] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.814963 29413 raft_consensus.cc:697] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.815356 28798 ts_tablet_manager.cc:1434] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.025s user 0.002s sys 0.000s
I20260618 13:59:29.816085 28799 raft_consensus.cc:359] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.815238 29413 consensus_queue.cc:237] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.816893 28799 raft_consensus.cc:385] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.817651 28799 raft_consensus.cc:740] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.819161 28798 tablet_bootstrap.cc:492] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:29.819479 28799 consensus_queue.cc:260] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.818869 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd2385e6752d49429422fc854539e9c6" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.820000 28407 raft_consensus.cc:2393] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:29.820451 28483 leader_election.cc:304] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.820899 29456 raft_consensus.cc:3060] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.822036 28799 ts_tablet_manager.cc:1434] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260618 13:59:29.823637 29456 raft_consensus.cc:2749] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.828595 28798 tablet_bootstrap.cc:654] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.829671 28799 tablet_bootstrap.cc:492] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:29.831833 28799 tablet_bootstrap.cc:654] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.833933 28797 tablet_bootstrap.cc:492] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:29.834182 28797 ts_tablet_manager.cc:1403] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.055s user 0.010s sys 0.011s
I20260618 13:59:29.835179 28797 raft_consensus.cc:359] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.835480 28797 raft_consensus.cc:385] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.835621 28797 raft_consensus.cc:740] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.835966 28797 consensus_queue.cc:260] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.836589 28797 ts_tablet_manager.cc:1434] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260618 13:59:29.836822 28618 consensus_peers.cc:597] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:29.836987 28797 tablet_bootstrap.cc:492] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:29.838917 28797 tablet_bootstrap.cc:654] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.841091 28798 tablet_bootstrap.cc:492] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:29.841321 28798 ts_tablet_manager.cc:1403] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.000s
I20260618 13:59:29.845474 29186 consensus_queue.cc:1048] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.842712 28798 raft_consensus.cc:359] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.850919 28798 raft_consensus.cc:385] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.851085 28798 raft_consensus.cc:740] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.851503 28798 consensus_queue.cc:260] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.852649 28798 ts_tablet_manager.cc:1434] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260618 13:59:29.853049 28798 tablet_bootstrap.cc:492] T 1829d1b69cdd4e748c5901c8e02cac2e P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:29.855263 28798 tablet_bootstrap.cc:654] T 1829d1b69cdd4e748c5901c8e02cac2e P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.855995 28799 tablet_bootstrap.cc:492] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:29.860687 28799 ts_tablet_manager.cc:1403] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.033s user 0.004s sys 0.000s
I20260618 13:59:29.861930 28797 tablet_bootstrap.cc:492] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:29.862210 28797 ts_tablet_manager.cc:1403] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.002s
I20260618 13:59:29.862202 28799 raft_consensus.cc:359] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.863906 28797 raft_consensus.cc:359] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.864255 28797 raft_consensus.cc:385] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.864442 28797 raft_consensus.cc:740] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.864904 28797 consensus_queue.cc:260] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.869974 28797 ts_tablet_manager.cc:1434] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20260618 13:59:29.870895 28799 raft_consensus.cc:385] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.869946 29489 raft_consensus.cc:493] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.871101 28799 raft_consensus.cc:740] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.871160 29489 raft_consensus.cc:515] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:29.871570 28797 tablet_bootstrap.cc:492] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:29.871927 29489 leader_election.cc:290] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:29.873183 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eea4d59935d40aa97dc10ec7469aa56" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.873865 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eea4d59935d40aa97dc10ec7469aa56" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:29.874068 28346 leader_election.cc:343] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:29.874708 28346 leader_election.cc:343] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.874851 28346 leader_election.cc:304] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.859088 29362 raft_consensus.cc:493] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.875602 29362 raft_consensus.cc:515] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.876452 29489 raft_consensus.cc:2749] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.876524 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d014b46a0f448b929cbc9287ead0a7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.876605 28797 tablet_bootstrap.cc:654] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:29.877051 28346 leader_election.cc:343] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:29.877635 28483 consensus_peers.cc:597] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c552d1316e3340f4bcfa39618ae309b8. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:29.878063 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d014b46a0f448b929cbc9287ead0a7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:29.878643 28346 leader_election.cc:343] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.878803 28346 leader_election.cc:304] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.879154 29489 raft_consensus.cc:2749] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.879896 29362 leader_election.cc:290] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.885303 28798 tablet_bootstrap.cc:492] T 1829d1b69cdd4e748c5901c8e02cac2e P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:29.885561 28798 ts_tablet_manager.cc:1403] T 1829d1b69cdd4e748c5901c8e02cac2e P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.033s user 0.005s sys 0.018s
I20260618 13:59:29.886469 28798 raft_consensus.cc:359] T 1829d1b69cdd4e748c5901c8e02cac2e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.886745 28798 raft_consensus.cc:385] T 1829d1b69cdd4e748c5901c8e02cac2e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.886890 28798 raft_consensus.cc:740] T 1829d1b69cdd4e748c5901c8e02cac2e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.887307 28798 consensus_queue.cc:260] T 1829d1b69cdd4e748c5901c8e02cac2e P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.888151 28798 ts_tablet_manager.cc:1434] T 1829d1b69cdd4e748c5901c8e02cac2e P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:29.888329 28799 consensus_queue.cc:260] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.888875 28798 tablet_bootstrap.cc:492] T b16f7d3c7837498e996bba1818110642 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:29.891131 28798 tablet_bootstrap.cc:654] T b16f7d3c7837498e996bba1818110642 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.894335 29456 raft_consensus.cc:493] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.894541 29456 raft_consensus.cc:515] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.895226 29456 leader_election.cc:290] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.895793 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d502d67826ff4d7a8b0f515e5d8b6d77" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.895828 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d502d67826ff4d7a8b0f515e5d8b6d77" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:29.896055 28410 raft_consensus.cc:2393] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:29.897002 28483 leader_election.cc:304] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:29.897068 28797 tablet_bootstrap.cc:492] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:29.897233 28797 ts_tablet_manager.cc:1403] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.001s
I20260618 13:59:29.897516 29456 raft_consensus.cc:3060] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.897993 28799 ts_tablet_manager.cc:1434] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.036s user 0.002s sys 0.000s
I20260618 13:59:29.898095 28797 raft_consensus.cc:359] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.898303 28797 raft_consensus.cc:385] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.898388 28797 raft_consensus.cc:740] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.898702 28797 consensus_queue.cc:260] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.900461 29456 raft_consensus.cc:2749] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:29.901957 28797 ts_tablet_manager.cc:1434] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260618 13:59:29.902370 28799 tablet_bootstrap.cc:492] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:29.902385 28797 tablet_bootstrap.cc:492] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:29.905292 28799 tablet_bootstrap.cc:654] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.909147 28797 tablet_bootstrap.cc:654] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.909740 29413 consensus_queue.cc:1048] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.912611 29413 consensus_queue.cc:1048] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.917812 29413 raft_consensus.cc:493] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.918000 29413 raft_consensus.cc:515] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.918627 29413 leader_election.cc:290] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.919576 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98b42d23832545ae81a7cdff19e4689f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:29.920306 28618 leader_election.cc:343] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.923482 28798 tablet_bootstrap.cc:492] T b16f7d3c7837498e996bba1818110642 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:29.923682 28798 ts_tablet_manager.cc:1403] T b16f7d3c7837498e996bba1818110642 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.035s user 0.003s sys 0.006s
I20260618 13:59:29.924845 28798 raft_consensus.cc:359] T b16f7d3c7837498e996bba1818110642 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.925091 28798 raft_consensus.cc:385] T b16f7d3c7837498e996bba1818110642 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.925204 28798 raft_consensus.cc:740] T b16f7d3c7837498e996bba1818110642 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.925552 28798 consensus_queue.cc:260] T b16f7d3c7837498e996bba1818110642 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.926297 28798 ts_tablet_manager.cc:1434] T b16f7d3c7837498e996bba1818110642 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:29.930795 28798 tablet_bootstrap.cc:492] T d7a97176f72349cda221baaea61d18ad P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:29.933467 28798 tablet_bootstrap.cc:654] T d7a97176f72349cda221baaea61d18ad P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.923655 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98b42d23832545ae81a7cdff19e4689f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:29.937672 28407 raft_consensus.cc:2468] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:29.938235 28617 leader_election.cc:304] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.938580 29402 raft_consensus.cc:2804] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:29.938673 29402 raft_consensus.cc:493] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:29.938798 29402 raft_consensus.cc:3060] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.941881 29402 raft_consensus.cc:515] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.943265 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98b42d23832545ae81a7cdff19e4689f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:29.943835 28618 leader_election.cc:343] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.944872 29402 leader_election.cc:290] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:29.944934 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98b42d23832545ae81a7cdff19e4689f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:29.945144 28407 raft_consensus.cc:3060] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:29.947995 28407 raft_consensus.cc:2468] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:29.948019 28798 tablet_bootstrap.cc:492] T d7a97176f72349cda221baaea61d18ad P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:29.948235 28798 ts_tablet_manager.cc:1403] T d7a97176f72349cda221baaea61d18ad P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.004s
I20260618 13:59:29.948812 28617 leader_election.cc:304] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:29.949525 29402 raft_consensus.cc:2804] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:29.949895 29402 raft_consensus.cc:697] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:29.950289 28798 raft_consensus.cc:359] T d7a97176f72349cda221baaea61d18ad P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.950877 28798 raft_consensus.cc:385] T d7a97176f72349cda221baaea61d18ad P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:29.950979 28798 raft_consensus.cc:740] T d7a97176f72349cda221baaea61d18ad P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:29.951287 28798 consensus_queue.cc:260] T d7a97176f72349cda221baaea61d18ad P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.951951 28798 ts_tablet_manager.cc:1434] T d7a97176f72349cda221baaea61d18ad P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:29.952293 28798 tablet_bootstrap.cc:492] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:29.954164 28798 tablet_bootstrap.cc:654] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.950626 29402 consensus_queue.cc:237] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.958124 28799 tablet_bootstrap.cc:492] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:29.958770 28799 ts_tablet_manager.cc:1403] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.058s user 0.004s sys 0.000s
I20260618 13:59:29.960632 29024 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 5e25027f423144b088dad955c7282921
I20260618 13:59:29.966063 29303 raft_consensus.cc:493] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:29.966069 29186 consensus_queue.cc:1048] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:29.966383 29303 raft_consensus.cc:515] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:29.967293 29303 leader_election.cc:290] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:29.968293 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6dd422c05324c989e29b20ae2afabad" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:29.969167 28618 leader_election.cc:343] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:29.970047 29024 catalog_manager.cc:6784] ReplaceTablet: tablet 5e25027f423144b088dad955c7282921 deleted and replaced by tablet a8da96fc621c47d4a444b5a39259f85d
W20260618 13:59:29.977101 28618 consensus_peers.cc:597] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:29.989521 28797 tablet_bootstrap.cc:492] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:29.995081 28797 ts_tablet_manager.cc:1403] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.093s user 0.010s sys 0.001s
I20260618 13:59:29.996919 28797 raft_consensus.cc:359] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:29.999956 28797 raft_consensus.cc:385] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.001320 28797 raft_consensus.cc:740] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.002365 28797 consensus_queue.cc:260] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:30.008927 28618 consensus_peers.cc:597] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:30.007803 28797 ts_tablet_manager.cc:1434] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.012s user 0.000s sys 0.002s
I20260618 13:59:30.009467 28797 tablet_bootstrap.cc:492] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:30.012449 28797 tablet_bootstrap.cc:654] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:29.967878 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6dd422c05324c989e29b20ae2afabad" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:30.013265 29020 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet d4667fe48f0f4eff9c6c8baabdab58d4
I20260618 13:59:30.013818 29020 catalog_manager.cc:6784] ReplaceTablet: tablet d4667fe48f0f4eff9c6c8baabdab58d4 deleted and replaced by tablet a662bb37fb7d4df49e61e524808b67a8
I20260618 13:59:30.022882 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e25027f423144b088dad955c7282921 with delete_type TABLET_DATA_DELETED (replaced by tablet a8da96fc621c47d4a444b5a39259f85d) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:30.023381 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet d4667fe48f0f4eff9c6c8baabdab58d4 with delete_type TABLET_DATA_DELETED (replaced by tablet a662bb37fb7d4df49e61e524808b67a8) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:30.024953 28798 tablet_bootstrap.cc:492] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:30.025177 28798 ts_tablet_manager.cc:1403] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.073s user 0.011s sys 0.043s
I20260618 13:59:30.026374 28798 raft_consensus.cc:359] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.026494 29017 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 97a9f949999446f09359629fe483bf67
I20260618 13:59:30.026984 28798 raft_consensus.cc:385] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.027166 28798 raft_consensus.cc:740] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.027529 28798 consensus_queue.cc:260] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.028227 28798 ts_tablet_manager.cc:1434] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260618 13:59:30.028373 28989 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 5e25027f423144b088dad955c7282921 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5e25027f423144b088dad955c7282921 already in progress: creating tablet
W20260618 13:59:30.028743 28989 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d4667fe48f0f4eff9c6c8baabdab58d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d4667fe48f0f4eff9c6c8baabdab58d4 already in progress: creating tablet
I20260618 13:59:30.028800 28798 tablet_bootstrap.cc:492] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:30.029583 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 97a9f949999446f09359629fe483bf67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:29 UTC) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:30.030267 28989 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 97a9f949999446f09359629fe483bf67 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 97a9f949999446f09359629fe483bf67 already in progress: creating tablet
I20260618 13:59:30.031809 28798 tablet_bootstrap.cc:654] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.033178 28797 tablet_bootstrap.cc:492] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:30.033420 28797 ts_tablet_manager.cc:1403] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.001s
I20260618 13:59:30.034530 28797 raft_consensus.cc:359] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.034809 28797 raft_consensus.cc:385] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.035147 28797 raft_consensus.cc:740] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.035516 28797 consensus_queue.cc:260] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.038515 28410 raft_consensus.cc:2468] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:30.039085 28617 leader_election.cc:304] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:30.039620 29303 raft_consensus.cc:2804] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:30.039765 29303 raft_consensus.cc:493] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:30.039878 29303 raft_consensus.cc:3060] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.040468 28799 raft_consensus.cc:359] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:30.040685 28799 raft_consensus.cc:385] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.040764 28799 raft_consensus.cc:740] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.041049 28799 consensus_queue.cc:260] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:30.043089 29303 raft_consensus.cc:515] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.043975 29402 consensus_queue.cc:1048] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:30.044030 28483 consensus_peers.cc:597] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 37a6764a5e024903848561c9935b1a43. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:30.044353 28797 ts_tablet_manager.cc:1434] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260618 13:59:30.044592 29303 leader_election.cc:290] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:30.045194 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6dd422c05324c989e29b20ae2afabad" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:30.045806 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6dd422c05324c989e29b20ae2afabad" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:30.046036 28409 raft_consensus.cc:3060] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.047542 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e25027f423144b088dad955c7282921 with delete_type TABLET_DATA_DELETED (replaced by tablet a8da96fc621c47d4a444b5a39259f85d) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:30.049022 28511 tablet_service.cc:1558] Processing DeleteTablet for tablet d4667fe48f0f4eff9c6c8baabdab58d4 with delete_type TABLET_DATA_DELETED (replaced by tablet a662bb37fb7d4df49e61e524808b67a8) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:30.049216 28516 tablet_service.cc:1558] Processing DeleteTablet for tablet 97a9f949999446f09359629fe483bf67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:29 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:30.049460 28409 raft_consensus.cc:2468] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
W20260618 13:59:30.049840 28618 leader_election.cc:343] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.050395 28617 leader_election.cc:304] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:30.050859 29303 raft_consensus.cc:2804] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:30.051050 29303 raft_consensus.cc:697] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:30.051376 29303 consensus_queue.cc:237] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.053957 28797 tablet_bootstrap.cc:492] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:30.056234 28373 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e25027f423144b088dad955c7282921 with delete_type TABLET_DATA_DELETED (replaced by tablet a8da96fc621c47d4a444b5a39259f85d) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:30.056633 28374 tablet_service.cc:1558] Processing DeleteTablet for tablet d4667fe48f0f4eff9c6c8baabdab58d4 with delete_type TABLET_DATA_DELETED (replaced by tablet a662bb37fb7d4df49e61e524808b67a8) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:30.056831 28376 tablet_service.cc:1558] Processing DeleteTablet for tablet 97a9f949999446f09359629fe483bf67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:29 UTC) from {username='slave'} at 127.0.0.1:38230
W20260618 13:59:30.057607 28618 consensus_peers.cc:597] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:30.062879 29007 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bf5e04eb43ed4cf493c0f6f08e22baeb consensus state has no leader
I20260618 13:59:30.063568 29007 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_e97ff5cd68d24da7a9a803e10b35dbe6" } new_table_name: "default.Table_c344ac246b4c4f11a16becbd5023032a" modify_external_catalogs: true
I20260618 13:59:30.063782 29011 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_e24e443c392040b0adf1e8c8eb581a10" } new_table_name: "default.Table_5de92d2ec52545f9946f35cf74ca4f01" modify_external_catalogs: true
I20260618 13:59:30.065348 29018 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_94aa74339cf547a080b6ede6a36b4bed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:30.065936 29018 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_94aa74339cf547a080b6ede6a36b4bed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:30.066201 29020 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_ea7b830f69254dc49cbc9e3c17c3be85" } new_table_name: "default.Table_8f6f0bc5df454e3e9e307c07a4b86c53" modify_external_catalogs: true
I20260618 13:59:30.068504 29456 consensus_queue.cc:1048] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.068670 29017 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_dc9c9ba6a1ce4446b3654e271445bd3a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:30.069346 29017 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc9c9ba6a1ce4446b3654e271445bd3a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:30.071759 29400 raft_consensus.cc:493] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.071978 29400 raft_consensus.cc:515] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.072767 29400 leader_election.cc:290] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:30.073032 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b8f273def3e4b7caa4d624cf87a106c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:30.073673 28618 leader_election.cc:343] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.073946 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b8f273def3e4b7caa4d624cf87a106c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:30.074149 28407 raft_consensus.cc:2468] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:30.074623 28617 leader_election.cc:304] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:30.074878 29402 raft_consensus.cc:2804] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:30.075006 29402 raft_consensus.cc:493] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:30.075114 29402 raft_consensus.cc:3060] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.075130 28797 tablet_bootstrap.cc:654] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.075973 29400 consensus_queue.cc:1048] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.077991 29402 raft_consensus.cc:515] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.079300 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b8f273def3e4b7caa4d624cf87a106c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:30.079494 28407 raft_consensus.cc:3060] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.079936 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b8f273def3e4b7caa4d624cf87a106c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:30.080633 29402 leader_election.cc:290] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:30.080977 28618 leader_election.cc:343] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:30.083292 28988 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 5e25027f423144b088dad955c7282921 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5e25027f423144b088dad955c7282921 already in progress: creating tablet
W20260618 13:59:30.083571 28988 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet d4667fe48f0f4eff9c6c8baabdab58d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d4667fe48f0f4eff9c6c8baabdab58d4 already in progress: creating tablet
W20260618 13:59:30.083770 28988 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 97a9f949999446f09359629fe483bf67 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 97a9f949999446f09359629fe483bf67 already in progress: creating tablet
I20260618 13:59:30.084267 28407 raft_consensus.cc:2468] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:30.084690 29025 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 015494374645469785f1f978eaf0e9a5
I20260618 13:59:30.085263 29025 catalog_manager.cc:6784] ReplaceTablet: tablet 015494374645469785f1f978eaf0e9a5 deleted and replaced by tablet 2e493f647a344cd581515a77230d765f
I20260618 13:59:30.085642 29025 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_76724583c0db40378d257d673a601220" } modify_external_catalogs: true
I20260618 13:59:30.085870 29025 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_76724583c0db40378d257d673a601220" } modify_external_catalogs: true
I20260618 13:59:30.088701 28373 tablet_service.cc:1558] Processing DeleteTablet for tablet 015494374645469785f1f978eaf0e9a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 2e493f647a344cd581515a77230d765f) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:30.088828 28547 raft_consensus.cc:3060] T b16f7d3c7837498e996bba1818110642 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.088891 28516 tablet_service.cc:1558] Processing DeleteTablet for tablet 015494374645469785f1f978eaf0e9a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 2e493f647a344cd581515a77230d765f) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:30.089620 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 015494374645469785f1f978eaf0e9a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 2e493f647a344cd581515a77230d765f) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:30.090237 28989 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 015494374645469785f1f978eaf0e9a5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 015494374645469785f1f978eaf0e9a5 already in progress: creating tablet
I20260618 13:59:30.092247 28617 leader_election.cc:304] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
W20260618 13:59:30.092396 28988 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 015494374645469785f1f978eaf0e9a5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 015494374645469785f1f978eaf0e9a5 already in progress: creating tablet
I20260618 13:59:30.093282 29402 raft_consensus.cc:2804] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:30.093567 29186 consensus_queue.cc:1048] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:30.094914 28988 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet bf5e04eb43ed4cf493c0f6f08e22baeb consensus state has no leader
I20260618 13:59:30.094388 29402 raft_consensus.cc:697] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:30.095669 29402 consensus_queue.cc:237] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.098582 29456 raft_consensus.cc:493] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.098783 29456 raft_consensus.cc:515] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:30.099382 29456 leader_election.cc:290] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:30.099965 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe750fc5d998448caeab0a6c808dfba7" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:30.100277 28407 raft_consensus.cc:2393] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:30.100533 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe750fc5d998448caeab0a6c808dfba7" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:30.101114 28484 leader_election.cc:304] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:30.101665 29456 raft_consensus.cc:3060] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.105001 29456 raft_consensus.cc:2749] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:30.106379 29186 consensus_queue.cc:1048] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.111222 29362 raft_consensus.cc:493] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.111421 29362 raft_consensus.cc:515] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.112190 29362 leader_election.cc:290] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:30.113031 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f42997416841a7b2c1ad347d2e9282" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:30.113626 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f42997416841a7b2c1ad347d2e9282" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:30.114331 28346 leader_election.cc:343] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:30.114594 28346 leader_election.cc:343] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.114697 28346 leader_election.cc:304] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:30.114920 29362 raft_consensus.cc:2749] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:30.122081 29019 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_fd6ef45aeec84b48b8fb7a54db4b4f81" } modify_external_catalogs: true
I20260618 13:59:30.122346 29019 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_fd6ef45aeec84b48b8fb7a54db4b4f81" } modify_external_catalogs: true
I20260618 13:59:30.125434 29023 catalog_manager.cc:5696] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } }
W20260618 13:59:30.126911 29007 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f54910015987484e95b295352d859315 consensus state has no leader
W20260618 13:59:30.127645 29011 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 56650c9e20e24a28821cca4279bfde58 consensus state has no leader
I20260618 13:59:30.126907 29023 catalog_manager.cc:5696] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:30.131946 29186 consensus_queue.cc:1048] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:30.155846 29020 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 450c68da26404767b01c5e8da3f6286d consensus state has no leader
I20260618 13:59:30.155723 29400 consensus_queue.cc:1048] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.160071 29456 raft_consensus.cc:493] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.160226 29456 raft_consensus.cc:515] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.160822 29456 leader_election.cc:290] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:30.161449 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc0adedecee4405e86b0a37b9e62e352" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:30.162729 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc0adedecee4405e86b0a37b9e62e352" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:30.163324 28484 leader_election.cc:304] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:30.163762 29456 raft_consensus.cc:2749] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:30.169335 29362 raft_consensus.cc:493] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.169489 29362 raft_consensus.cc:515] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.170068 29362 leader_election.cc:290] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:30.171039 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15180aefba0544329daee74308ef5010" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:30.171042 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15180aefba0544329daee74308ef5010" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:30.171609 28346 leader_election.cc:343] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:30.175760 28346 leader_election.cc:343] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.175804 29303 consensus_queue.cc:1048] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.175917 28346 leader_election.cc:304] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:30.177220 29362 raft_consensus.cc:2749] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:30.182852 29456 consensus_queue.cc:1048] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.191887 29456 consensus_queue.cc:1048] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.203380 28799 ts_tablet_manager.cc:1434] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.191s user 0.002s sys 0.000s
W20260618 13:59:30.204591 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 5e25027f423144b088dad955c7282921 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5e25027f423144b088dad955c7282921 already in progress: creating tablet
W20260618 13:59:30.205803 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d4667fe48f0f4eff9c6c8baabdab58d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d4667fe48f0f4eff9c6c8baabdab58d4 already in progress: creating tablet
I20260618 13:59:30.208478 29402 consensus_queue.cc:1048] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.212105 29456 raft_consensus.cc:493] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.212350 29456 raft_consensus.cc:515] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:30.212558 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 97a9f949999446f09359629fe483bf67 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 97a9f949999446f09359629fe483bf67 already in progress: creating tablet
I20260618 13:59:30.215754 29456 leader_election.cc:290] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:30.216619 29401 consensus_queue.cc:1048] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.218016 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6b01b897134d5bbd6a7d93b82ddb5e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:30.203632 29489 raft_consensus.cc:493] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.213657 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6b01b897134d5bbd6a7d93b82ddb5e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:30.224329 29489 raft_consensus.cc:515] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.224761 28483 leader_election.cc:304] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:30.225044 29489 leader_election.cc:290] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:30.225210 29221 raft_consensus.cc:2749] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:30.226253 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be59f988f0e34b9480f859cf88b9c223" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:30.226990 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be59f988f0e34b9480f859cf88b9c223" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:30.228088 28346 leader_election.cc:343] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:30.228327 28346 leader_election.cc:343] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.228470 28346 leader_election.cc:304] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:30.229089 29489 raft_consensus.cc:2749] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:30.229265 29522 raft_consensus.cc:493] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.229404 29522 raft_consensus.cc:515] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:30.229970 29522 leader_election.cc:290] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:30.232019 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be00bfa9bc1a4b83b18bfad05ba1495e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:30.232641 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be00bfa9bc1a4b83b18bfad05ba1495e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:30.232740 28346 leader_election.cc:343] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:30.232424 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 015494374645469785f1f978eaf0e9a5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 015494374645469785f1f978eaf0e9a5 already in progress: creating tablet
W20260618 13:59:30.233147 28346 leader_election.cc:343] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.233280 28346 leader_election.cc:304] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:30.233633 29522 raft_consensus.cc:2749] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:30.234112 28798 tablet_bootstrap.cc:492] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:30.234319 28798 ts_tablet_manager.cc:1403] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.206s user 0.021s sys 0.024s
I20260618 13:59:30.235193 29011 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_3ef2ff183a6044e2a9e069a4eac133b8" } modify_external_catalogs: true
I20260618 13:59:30.235374 29011 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_3ef2ff183a6044e2a9e069a4eac133b8" } modify_external_catalogs: true
I20260618 13:59:30.235406 28798 raft_consensus.cc:359] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.235592 28798 raft_consensus.cc:385] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.235674 28798 raft_consensus.cc:740] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.236837 29007 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_2dc7e2d92c504d7f91ac02de7c2bdee4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:30.237426 29007 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2dc7e2d92c504d7f91ac02de7c2bdee4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:30.239199 29020 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_9a78338c3e414b0abea1281082202f57" } modify_external_catalogs: true
I20260618 13:59:30.239405 29020 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_9a78338c3e414b0abea1281082202f57" } modify_external_catalogs: true
I20260618 13:59:30.240984 28799 tablet_bootstrap.cc:492] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:30.242616 28483 consensus_peers.cc:597] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 75b9c0754beb4e0e8ed3a7c1e9f368a9. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:30.244248 29456 raft_consensus.cc:493] T 1829d1b69cdd4e748c5901c8e02cac2e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.245127 29456 raft_consensus.cc:515] T 1829d1b69cdd4e748c5901c8e02cac2e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.245553 28544 raft_consensus.cc:3060] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.245976 29456 leader_election.cc:290] T 1829d1b69cdd4e748c5901c8e02cac2e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:30.246454 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1829d1b69cdd4e748c5901c8e02cac2e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:30.246759 28798 consensus_queue.cc:260] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.247128 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1829d1b69cdd4e748c5901c8e02cac2e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:30.247697 28484 leader_election.cc:304] T 1829d1b69cdd4e748c5901c8e02cac2e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:30.248346 29456 raft_consensus.cc:2749] T 1829d1b69cdd4e748c5901c8e02cac2e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:30.249157 28799 tablet_bootstrap.cc:654] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.258085 28546 raft_consensus.cc:3060] T d7a97176f72349cda221baaea61d18ad P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.268276 29230 raft_consensus.cc:493] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.268471 29230 raft_consensus.cc:515] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.269084 29230 leader_election.cc:290] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:30.269767 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "751cd9bd961b4f56a588ed02a05f0212" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:30.270955 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "751cd9bd961b4f56a588ed02a05f0212" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:30.272313 28484 leader_election.cc:304] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:30.273520 29400 consensus_queue.cc:1048] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.275908 28798 ts_tablet_manager.cc:1434] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.026s user 0.001s sys 0.001s
I20260618 13:59:30.276260 28798 tablet_bootstrap.cc:492] T 9b75ece59d924e3db49e002bf0faa89c P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:30.276517 29018 master_service.cc:960] ReplaceTablet: received request to replace tablet ca2ee746c5024cfc9b31c470f9616e7d from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:30.277058 29230 raft_consensus.cc:2749] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:30.291124 29025 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 69d5e75b4a8247b29d69d07fad5ca5db
I20260618 13:59:30.294044 29017 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_473a0c23203b404ea8e5ee0aed201e7b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:30.294687 29017 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_473a0c23203b404ea8e5ee0aed201e7b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:30.297348 28596 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:30.297423 28798 tablet_bootstrap.cc:654] T 9b75ece59d924e3db49e002bf0faa89c P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.307060 29025 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_539ac216b6f4478eb994218d7d7980a4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:30.307804 29025 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_539ac216b6f4478eb994218d7d7980a4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:30.309676 29400 raft_consensus.cc:493] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.309840 29400 raft_consensus.cc:515] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.310539 29400 leader_election.cc:290] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:30.310602 28797 tablet_bootstrap.cc:492] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:30.310595 29026 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_12d6cc95071949c6a7a297cd08b5f898" } new_table_name: "default.Table_f44be481def649db8daed0bf151a258c" modify_external_catalogs: true
I20260618 13:59:30.310763 28797 ts_tablet_manager.cc:1403] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.257s user 0.024s sys 0.017s
I20260618 13:59:30.311051 29400 raft_consensus.cc:493] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.311213 29400 raft_consensus.cc:515] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.311554 28797 raft_consensus.cc:359] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.311741 28797 raft_consensus.cc:385] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.311811 29400 leader_election.cc:290] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:30.312291 28797 raft_consensus.cc:740] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.312510 28797 consensus_queue.cc:260] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.313037 28797 ts_tablet_manager.cc:1434] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:30.313361 28797 tablet_bootstrap.cc:492] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:30.315788 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512a9e63ba324a30b8d8c37e585ebdf8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:30.315845 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc36fbea853a4d4e9f61909200239aed" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:30.316394 28618 leader_election.cc:343] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:30.316653 28618 leader_election.cc:343] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.317979 28798 tablet_bootstrap.cc:492] T 9b75ece59d924e3db49e002bf0faa89c P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:30.318112 28798 ts_tablet_manager.cc:1403] T 9b75ece59d924e3db49e002bf0faa89c P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.042s user 0.001s sys 0.002s
I20260618 13:59:30.318894 28798 raft_consensus.cc:359] T 9b75ece59d924e3db49e002bf0faa89c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.319067 28798 raft_consensus.cc:385] T 9b75ece59d924e3db49e002bf0faa89c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.319144 28798 raft_consensus.cc:740] T 9b75ece59d924e3db49e002bf0faa89c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.319489 28797 tablet_bootstrap.cc:654] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.323504 28798 consensus_queue.cc:260] T 9b75ece59d924e3db49e002bf0faa89c P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.326498 28798 ts_tablet_manager.cc:1434] T 9b75ece59d924e3db49e002bf0faa89c P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260618 13:59:30.330685 29400 raft_consensus.cc:493] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.330897 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet 69d5e75b4a8247b29d69d07fad5ca5db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:30.330866 29400 raft_consensus.cc:515] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.331295 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet d502d67826ff4d7a8b0f515e5d8b6d77 with delete_type TABLET_DATA_DELETED (replaced by tablet 657e55fa53c447babfe91b1582806e6b) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:30.331566 29400 leader_election.cc:290] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:30.331770 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 69d5e75b4a8247b29d69d07fad5ca5db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:30.332247 29566 tablet_replica.cc:333] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:30.332454 29566 raft_consensus.cc:2243] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:30.332760 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet 2188ebf1727240cd886ba7662f8fcf43 with delete_type TABLET_DATA_DELETED (replaced by tablet c0a42aa779ae452e9420d730bf8d7d55) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:30.332863 29566 raft_consensus.cc:2272] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:30.333225 28516 tablet_service.cc:1558] Processing DeleteTablet for tablet edc72a06ee8e4420b07a17d2feb3fcc2 with delete_type TABLET_DATA_DELETED (replaced by tablet aa8cf927bed9437086efd8e0f21ba26a) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:30.333421 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet 4a01dfb4d59549399c94640f16045555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:30.333792 28798 tablet_bootstrap.cc:492] T f89c9954d48e407c96c294c26644e25c P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:30.334393 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118259402d7d48dda381cdd18eec0f1b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:30.334775 29566 ts_tablet_manager.cc:1916] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:30.335125 28618 consensus_peers.cc:597] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:30.335355 28618 leader_election.cc:343] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:30.335561 28989 catalog_manager.cc:5106] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): alter failed for tablet 69d5e75b4a8247b29d69d07fad5ca5db (table default.Table_76724583c0db40378d257d673a601220 [id=5278d8eb60e4448ebae7afc3edb0db8e]): Illegal state: Tablet not RUNNING: STOPPING
I20260618 13:59:30.336782 29570 tablet_replica.cc:333] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:30.337107 29570 raft_consensus.cc:2243] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:30.337384 29570 raft_consensus.cc:2272] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:30.338042 29570 ts_tablet_manager.cc:1916] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:30.341902 28373 tablet_service.cc:1558] Processing DeleteTablet for tablet 69d5e75b4a8247b29d69d07fad5ca5db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:30.341929 28799 tablet_bootstrap.cc:492] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:30.343194 29572 tablet_replica.cc:333] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:30.343448 29572 raft_consensus.cc:2243] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:30.344216 28799 ts_tablet_manager.cc:1403] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.112s user 0.005s sys 0.001s
I20260618 13:59:30.344494 29572 raft_consensus.cc:2272] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:30.344995 28798 tablet_bootstrap.cc:654] T f89c9954d48e407c96c294c26644e25c P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.345366 29572 ts_tablet_manager.cc:1916] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:30.350714 29566 ts_tablet_manager.cc:1929] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:30.351446 28799 raft_consensus.cc:359] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.351691 28799 raft_consensus.cc:385] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.352195 28799 raft_consensus.cc:740] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.352450 28799 consensus_queue.cc:260] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.353802 29401 raft_consensus.cc:493] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.353974 29401 raft_consensus.cc:515] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.354768 29401 leader_election.cc:290] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:30.355487 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f3d2f236fd44ab8af0524ea282cfa94" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:30.356177 28618 leader_election.cc:343] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.361056 29221 raft_consensus.cc:493] T 9b75ece59d924e3db49e002bf0faa89c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.361333 29221 raft_consensus.cc:515] T 9b75ece59d924e3db49e002bf0faa89c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.363770 29221 leader_election.cc:290] T 9b75ece59d924e3db49e002bf0faa89c P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:30.365619 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b75ece59d924e3db49e002bf0faa89c" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:30.365731 28376 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b88ed7cf79c4929a8704f66691cbed4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:30.365916 28373 tablet_service.cc:1558] Processing DeleteTablet for tablet 233040a16b0d4b61b977ca57c1afec2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:30.366189 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b75ece59d924e3db49e002bf0faa89c" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:30.366869 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet c84189a4b1194c87bbdf81f6e0ed56e4 with delete_type TABLET_DATA_DELETED (replaced by tablet be00bfa9bc1a4b83b18bfad05ba1495e) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:30.367364 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet edc72a06ee8e4420b07a17d2feb3fcc2 with delete_type TABLET_DATA_DELETED (replaced by tablet aa8cf927bed9437086efd8e0f21ba26a) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:30.367892 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 4a01dfb4d59549399c94640f16045555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:30.368366 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 233040a16b0d4b61b977ca57c1afec2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:30.368863 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 41107eb1d9164a9c98060003b4027412 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f5d4296952e48a48756e6f27140da87) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:30.369283 28799 ts_tablet_manager.cc:1434] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260618 13:59:30.369350 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b88ed7cf79c4929a8704f66691cbed4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:30.369246 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc36fbea853a4d4e9f61909200239aed" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:30.370467 28730 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
W20260618 13:59:30.371397 28989 catalog_manager.cc:5100] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): alter failed for tablet 69d5e75b4a8247b29d69d07fad5ca5db (table default.Table_76724583c0db40378d257d673a601220 [id=5278d8eb60e4448ebae7afc3edb0db8e]),no further retry: Illegal state: Tablet not RUNNING: STOPPED
I20260618 13:59:30.375659 28798 tablet_bootstrap.cc:492] T f89c9954d48e407c96c294c26644e25c P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:30.376135 29566 log.cc:1199] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/69d5e75b4a8247b29d69d07fad5ca5db
I20260618 13:59:30.376456 28798 ts_tablet_manager.cc:1403] T f89c9954d48e407c96c294c26644e25c P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.043s user 0.007s sys 0.002s
I20260618 13:59:30.376678 29566 ts_tablet_manager.cc:1950] T 69d5e75b4a8247b29d69d07fad5ca5db P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:30.369711 29599 tablet.cc:1724] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:30.378747 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 69d5e75b4a8247b29d69d07fad5ca5db (table default.Table_76724583c0db40378d257d673a601220 [id=5278d8eb60e4448ebae7afc3edb0db8e]) successfully deleted
I20260618 13:59:30.380064 29566 tablet_replica.cc:333] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:30.380350 29566 raft_consensus.cc:2243] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:30.380650 29566 raft_consensus.cc:2272] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:30.381996 28798 raft_consensus.cc:359] T f89c9954d48e407c96c294c26644e25c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:30.382359 28798 raft_consensus.cc:385] T f89c9954d48e407c96c294c26644e25c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.382506 28798 raft_consensus.cc:740] T f89c9954d48e407c96c294c26644e25c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.383489 29566 ts_tablet_manager.cc:1916] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:30.383836 28798 consensus_queue.cc:260] T f89c9954d48e407c96c294c26644e25c P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:30.385735 28798 ts_tablet_manager.cc:1434] T f89c9954d48e407c96c294c26644e25c P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260618 13:59:30.390166 29011 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet f3ba9186b3ff47d5a2b158fef619686e
I20260618 13:59:30.390282 28798 tablet_bootstrap.cc:492] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:30.385260 29570 ts_tablet_manager.cc:1929] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:30.391472 29570 log.cc:1199] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/69d5e75b4a8247b29d69d07fad5ca5db
I20260618 13:59:30.391764 29023 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_152a483501b54a67bbdc0cae4959d452"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:30.392412 29023 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_152a483501b54a67bbdc0cae4959d452 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:30.394258 29007 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_5592094c6a024bd6ac90af5c18b0daab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:30.394660 29570 ts_tablet_manager.cc:1950] T 69d5e75b4a8247b29d69d07fad5ca5db P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
W20260618 13:59:30.394832 29007 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5592094c6a024bd6ac90af5c18b0daab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:30.394829 28458 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:30.396680 29011 master_service.cc:960] ReplaceTablet: received request to replace tablet 897bf6d6bbee4ad89c296325d94a3e53 from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:30.396943 29572 ts_tablet_manager.cc:1929] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:30.397166 28373 tablet_service.cc:1558] Processing DeleteTablet for tablet f3ba9186b3ff47d5a2b158fef619686e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:30.397392 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512a9e63ba324a30b8d8c37e585ebdf8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:30.397725 28410 raft_consensus.cc:2468] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:30.397190 29572 log.cc:1199] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/69d5e75b4a8247b29d69d07fad5ca5db
I20260618 13:59:30.398587 29572 ts_tablet_manager.cc:1950] T 69d5e75b4a8247b29d69d07fad5ca5db P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:30.399461 29570 tablet_replica.cc:333] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:30.399418 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118259402d7d48dda381cdd18eec0f1b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:30.399700 29570 raft_consensus.cc:2243] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:30.399829 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet f3ba9186b3ff47d5a2b158fef619686e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:30.400457 28799 tablet_bootstrap.cc:492] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:30.400410 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet f3ba9186b3ff47d5a2b158fef619686e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:30.400600 28798 tablet_bootstrap.cc:654] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.400630 28408 raft_consensus.cc:2468] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:30.400883 28411 raft_consensus.cc:2393] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:30.399935 28407 raft_consensus.cc:2468] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:30.401464 28483 leader_election.cc:304] T 9b75ece59d924e3db49e002bf0faa89c P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:30.401600 29570 raft_consensus.cc:2272] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:30.402172 29570 ts_tablet_manager.cc:1916] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:30.402560 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 69d5e75b4a8247b29d69d07fad5ca5db (table default.Table_76724583c0db40378d257d673a601220 [id=5278d8eb60e4448ebae7afc3edb0db8e]) successfully deleted
I20260618 13:59:30.404891 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 69d5e75b4a8247b29d69d07fad5ca5db (table default.Table_76724583c0db40378d257d673a601220 [id=5278d8eb60e4448ebae7afc3edb0db8e]) successfully deleted
I20260618 13:59:30.402356 29221 raft_consensus.cc:3060] T 9b75ece59d924e3db49e002bf0faa89c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.405993 29566 ts_tablet_manager.cc:1929] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:30.406553 29566 log.cc:1199] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/c84189a4b1194c87bbdf81f6e0ed56e4
I20260618 13:59:30.406685 28617 leader_election.cc:304] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:30.407038 29566 ts_tablet_manager.cc:1950] T c84189a4b1194c87bbdf81f6e0ed56e4 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:30.407133 29402 consensus_queue.cc:1048] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.407552 29402 consensus_queue.cc:1048] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.407676 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet c84189a4b1194c87bbdf81f6e0ed56e4 (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]) successfully deleted
I20260618 13:59:30.407933 29402 consensus_queue.cc:1048] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.408242 29402 consensus_queue.cc:1048] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.408429 29221 raft_consensus.cc:2749] T 9b75ece59d924e3db49e002bf0faa89c P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:30.408591 29402 raft_consensus.cc:2804] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:30.408703 29402 raft_consensus.cc:493] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:30.408800 29402 raft_consensus.cc:3060] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.407059 29401 consensus_queue.cc:1048] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260618 13:59:30.413072 29402 raft_consensus.cc:515] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.414681 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512a9e63ba324a30b8d8c37e585ebdf8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:30.415125 29402 leader_election.cc:290] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:30.415450 28618 leader_election.cc:343] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.416781 29018 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet ca2ee746c5024cfc9b31c470f9616e7d
I20260618 13:59:30.417523 29018 catalog_manager.cc:6784] ReplaceTablet: tablet ca2ee746c5024cfc9b31c470f9616e7d deleted and replaced by tablet 60b6216474b74a14af6abf977e312b77
I20260618 13:59:30.417912 29018 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_0e9f574416d84a0f826fa9a1fcc56210" } modify_external_catalogs: true
I20260618 13:59:30.418174 29018 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_0e9f574416d84a0f826fa9a1fcc56210" } modify_external_catalogs: true
I20260618 13:59:30.419292 29017 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_f645609880ff40afb013281b28341bca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:30.419919 29017 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f645609880ff40afb013281b28341bca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:30.424136 29570 ts_tablet_manager.cc:1929] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:30.424301 29570 log.cc:1199] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/d502d67826ff4d7a8b0f515e5d8b6d77
I20260618 13:59:30.424719 29570 ts_tablet_manager.cc:1950] T d502d67826ff4d7a8b0f515e5d8b6d77 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:30.424808 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f3d2f236fd44ab8af0524ea282cfa94" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:30.425020 28407 raft_consensus.cc:2468] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:30.427582 28373 tablet_service.cc:1558] Processing DeleteTablet for tablet ca2ee746c5024cfc9b31c470f9616e7d with delete_type TABLET_DATA_DELETED (replaced by tablet 60b6216474b74a14af6abf977e312b77) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:30.427822 29572 tablet_replica.cc:333] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:30.428872 28797 tablet_bootstrap.cc:492] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:30.429039 28797 ts_tablet_manager.cc:1403] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.116s user 0.005s sys 0.004s
I20260618 13:59:30.429744 28797 raft_consensus.cc:359] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:30.429924 28797 raft_consensus.cc:385] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.429996 28797 raft_consensus.cc:740] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.430462 28797 consensus_queue.cc:260] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:30.431432 28797 ts_tablet_manager.cc:1434] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 13:59:30.433097 28797 tablet_bootstrap.cc:492] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:30.435024 28797 tablet_bootstrap.cc:654] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.435786 29570 tablet_replica.cc:333] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:30.436313 29570 raft_consensus.cc:2243] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:30.436798 29570 raft_consensus.cc:2272] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:30.442792 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet d502d67826ff4d7a8b0f515e5d8b6d77 (table default.Table_b5800aca80594ec1bd3a86ec47dbfc64 [id=2710734517a24ad6baf0d47bf23e9bdf]) successfully deleted
I20260618 13:59:30.442803 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet ca2ee746c5024cfc9b31c470f9616e7d with delete_type TABLET_DATA_DELETED (replaced by tablet 60b6216474b74a14af6abf977e312b77) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:30.442963 29570 ts_tablet_manager.cc:1916] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:30.446030 28799 tablet_bootstrap.cc:654] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.447578 29572 raft_consensus.cc:2243] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:30.447862 29572 raft_consensus.cc:2272] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:30.435907 28617 leader_election.cc:304] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:30.448920 28617 leader_election.cc:304] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:30.449596 28617 leader_election.cc:304] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:30.450565 29413 raft_consensus.cc:2804] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:30.450668 29413 raft_consensus.cc:493] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:30.450770 29413 raft_consensus.cc:3060] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.452637 29597 tablet.cc:1724] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:30.453500 29413 raft_consensus.cc:515] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.454115 29186 raft_consensus.cc:2804] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:30.454214 29186 raft_consensus.cc:493] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:30.454313 29186 raft_consensus.cc:3060] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.456370 29401 consensus_queue.cc:1048] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.454022 28799 tablet_bootstrap.cc:492] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:30.456795 28799 ts_tablet_manager.cc:1403] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.087s user 0.005s sys 0.000s
I20260618 13:59:30.456847 29401 raft_consensus.cc:2804] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:30.456954 29401 raft_consensus.cc:493] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:30.457067 29401 raft_consensus.cc:3060] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.457008 29186 raft_consensus.cc:515] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.457640 29186 leader_election.cc:290] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:30.458170 29572 ts_tablet_manager.cc:1916] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:30.458309 28799 raft_consensus.cc:359] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.458745 28799 raft_consensus.cc:385] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.458891 28799 raft_consensus.cc:740] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.459272 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512a9e63ba324a30b8d8c37e585ebdf8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:30.459234 28799 consensus_queue.cc:260] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.459465 28408 raft_consensus.cc:3060] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.459823 28799 ts_tablet_manager.cc:1434] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:30.460151 28799 tablet_bootstrap.cc:492] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:30.462165 28799 tablet_bootstrap.cc:654] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.463126 29401 raft_consensus.cc:515] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.463342 29413 leader_election.cc:290] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:30.439267 28618 consensus_peers.cc:597] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:30.442175 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet ca2ee746c5024cfc9b31c470f9616e7d with delete_type TABLET_DATA_DELETED (replaced by tablet 60b6216474b74a14af6abf977e312b77) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:30.464594 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc36fbea853a4d4e9f61909200239aed" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:30.464951 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc36fbea853a4d4e9f61909200239aed" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:30.465286 28403 raft_consensus.cc:3060] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.466600 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118259402d7d48dda381cdd18eec0f1b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:30.467161 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118259402d7d48dda381cdd18eec0f1b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:30.467495 28402 raft_consensus.cc:3060] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.468591 28548 raft_consensus.cc:3060] T f89c9954d48e407c96c294c26644e25c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:30.472626 28618 leader_election.cc:343] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:30.472994 28618 leader_election.cc:343] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:30.473219 28618 consensus_peers.cc:597] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d502d67826ff4d7a8b0f515e5d8b6d77. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:30.473543 29186 consensus_queue.cc:1048] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.478014 28408 raft_consensus.cc:2468] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:30.478622 28617 leader_election.cc:304] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:30.479060 29186 raft_consensus.cc:2804] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:30.479198 29186 raft_consensus.cc:697] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:30.479111 29023 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_df7ccb8b212342b5ad9837bf98607ec9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:30.479452 29186 consensus_queue.cc:237] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:30.479725 29023 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df7ccb8b212342b5ad9837bf98607ec9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:30.480197 29007 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_caa35bd6fefa4c5bab90b4b7f1ec48d9" } new_table_name: "default.Table_c20f026cc38841858fb9df9a0a01f852" modify_external_catalogs: true
I20260618 13:59:30.481017 28403 raft_consensus.cc:2468] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:30.481705 28617 leader_election.cc:304] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:30.482092 29604 raft_consensus.cc:2804] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:30.482228 29604 raft_consensus.cc:697] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:30.482595 29604 consensus_queue.cc:237] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.486792 28402 raft_consensus.cc:2468] T 118259402d7d48dda381cdd18eec0f1b P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:30.487493 28617 leader_election.cc:304] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:30.487915 29413 raft_consensus.cc:2804] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:30.488246 29413 raft_consensus.cc:697] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:30.488319 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f3d2f236fd44ab8af0524ea282cfa94" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:30.488615 29413 consensus_queue.cc:237] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:30.488911 28618 leader_election.cc:343] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.492624 29401 leader_election.cc:290] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:30.494287 29619 tablet.cc:1724] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:30.496230 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f3d2f236fd44ab8af0524ea282cfa94" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:30.496860 28402 raft_consensus.cc:3060] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.501688 28402 raft_consensus.cc:2468] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:30.502224 28617 leader_election.cc:304] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:30.503201 29402 raft_consensus.cc:2804] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:30.503403 29402 raft_consensus.cc:697] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:30.503937 29402 consensus_queue.cc:237] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:30.507299 28618 consensus_peers.cc:597] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:30.517060 29570 ts_tablet_manager.cc:1929] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:30.524343 28797 tablet_bootstrap.cc:492] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:30.524621 28797 ts_tablet_manager.cc:1403] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.093s user 0.003s sys 0.001s
I20260618 13:59:30.525821 28797 raft_consensus.cc:359] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.526069 29401 consensus_queue.cc:1048] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.529129 28797 raft_consensus.cc:385] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.529594 28797 raft_consensus.cc:740] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.530398 28797 consensus_queue.cc:260] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.532233 28797 ts_tablet_manager.cc:1434] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260618 13:59:30.533705 28797 tablet_bootstrap.cc:492] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:30.534330 29570 log.cc:1199] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/2188ebf1727240cd886ba7662f8fcf43
I20260618 13:59:30.536379 28797 tablet_bootstrap.cc:654] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:30.536082 28483 consensus_peers.cc:597] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac58e939730c4af681c2f08bcfbb6602. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:30.541241 29570 ts_tablet_manager.cc:1950] T 2188ebf1727240cd886ba7662f8fcf43 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:30.541890 29566 tablet_replica.cc:333] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:30.545504 29401 raft_consensus.cc:493] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.546156 29401 raft_consensus.cc:515] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.546958 29401 leader_election.cc:290] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:30.547883 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fae928833e44d2ba860d6184d4c0142" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:30.548234 28410 raft_consensus.cc:2468] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:30.548768 28617 leader_election.cc:304] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:30.549098 29401 raft_consensus.cc:2804] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:30.549407 29566 raft_consensus.cc:2243] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:30.549813 29401 raft_consensus.cc:493] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:30.549913 29401 raft_consensus.cc:3060] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.550366 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 2188ebf1727240cd886ba7662f8fcf43 (table default.Table_fac3fdd9b466497683bc00300a85fed1 [id=a4ac38531af34b4681abba96b42ffb7f]) successfully deleted
I20260618 13:59:30.550513 28798 tablet_bootstrap.cc:492] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:30.550679 28798 ts_tablet_manager.cc:1403] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.161s user 0.016s sys 0.022s
I20260618 13:59:30.551770 28798 raft_consensus.cc:359] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.552017 28798 raft_consensus.cc:385] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.552134 28798 raft_consensus.cc:740] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.552666 28798 consensus_queue.cc:260] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.554410 28798 ts_tablet_manager.cc:1434] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:30.562637 29570 tablet_replica.cc:333] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:30.562664 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fae928833e44d2ba860d6184d4c0142" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:30.563078 29570 raft_consensus.cc:2243] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:30.563562 29303 consensus_queue.cc:1048] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260618 13:59:30.563562 28618 leader_election.cc:343] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:30.565685 28618 consensus_peers.cc:597] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:30.574265 29566 raft_consensus.cc:2272] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:30.575132 29401 raft_consensus.cc:515] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.576484 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fae928833e44d2ba860d6184d4c0142" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:30.577124 28618 leader_election.cc:343] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.577340 29401 leader_election.cc:290] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:30.578105 28798 tablet_bootstrap.cc:492] T e86aeb21427e4859a5266f4451469b25 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:30.580308 28406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fae928833e44d2ba860d6184d4c0142" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:30.580613 28406 raft_consensus.cc:3060] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.581421 29566 ts_tablet_manager.cc:1916] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:30.587517 28799 tablet_bootstrap.cc:492] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:30.587700 28799 ts_tablet_manager.cc:1403] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.128s user 0.015s sys 0.017s
I20260618 13:59:30.588611 28799 raft_consensus.cc:359] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.588806 28799 raft_consensus.cc:385] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.588915 28799 raft_consensus.cc:740] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.589177 28799 consensus_queue.cc:260] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.589772 28799 ts_tablet_manager.cc:1434] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:30.590092 28799 tablet_bootstrap.cc:492] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:30.592978 28799 tablet_bootstrap.cc:654] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.589557 29413 consensus_queue.cc:1048] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.604380 28549 raft_consensus.cc:3060] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.580003 28798 tablet_bootstrap.cc:654] T e86aeb21427e4859a5266f4451469b25 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.611508 28797 tablet_bootstrap.cc:492] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:30.613071 28799 tablet_bootstrap.cc:492] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:30.613337 28799 ts_tablet_manager.cc:1403] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.007s
I20260618 13:59:30.613796 29017 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_3fa48c06a84c475faa51492f5328d840"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:30.614454 28799 raft_consensus.cc:359] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:30.614585 29017 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3fa48c06a84c475faa51492f5328d840 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:30.614779 28799 raft_consensus.cc:385] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.614907 28799 raft_consensus.cc:740] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.615268 28799 consensus_queue.cc:260] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.617149 28799 ts_tablet_manager.cc:1434] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260618 13:59:30.618258 28799 tablet_bootstrap.cc:492] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:30.620055 29221 consensus_queue.cc:1048] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260618 13:59:30.620626 28799 tablet_bootstrap.cc:654] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.621452 29570 raft_consensus.cc:2272] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:30.622471 29570 ts_tablet_manager.cc:1916] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:30.621337 28618 consensus_peers.cc:597] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:30.625396 28373 tablet_service.cc:1511] Processing CreateTablet for tablet c871e7c8f843486da7ba40145c0fed19 (DEFAULT_TABLE table=default.Table_c6e2f443374a47d2a787049499a0b168 [id=e875c5c4a0004e8287426403b0b9c626]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.630985 28798 tablet_bootstrap.cc:492] T e86aeb21427e4859a5266f4451469b25 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:30.631348 28798 ts_tablet_manager.cc:1403] T e86aeb21427e4859a5266f4451469b25 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.054s user 0.004s sys 0.000s
I20260618 13:59:30.632411 28798 raft_consensus.cc:359] T e86aeb21427e4859a5266f4451469b25 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.632754 28798 raft_consensus.cc:385] T e86aeb21427e4859a5266f4451469b25 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.632922 28798 raft_consensus.cc:740] T e86aeb21427e4859a5266f4451469b25 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.633287 28798 consensus_queue.cc:260] T e86aeb21427e4859a5266f4451469b25 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.633965 28798 ts_tablet_manager.cc:1434] T e86aeb21427e4859a5266f4451469b25 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:30.634002 28797 ts_tablet_manager.cc:1403] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.100s user 0.002s sys 0.004s
I20260618 13:59:30.634641 28798 tablet_bootstrap.cc:492] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:30.636946 29023 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_768d783dc0a7416686369250c3cd855c" } new_table_name: "default.Table_16516271526f49bf92301a70916453c1" modify_external_catalogs: true
I20260618 13:59:30.638573 28799 tablet_bootstrap.cc:492] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:30.638831 28799 ts_tablet_manager.cc:1403] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.001s
I20260618 13:59:30.639753 28799 raft_consensus.cc:359] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.640515 28799 raft_consensus.cc:385] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.640583 28798 tablet_bootstrap.cc:654] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.640738 28799 raft_consensus.cc:740] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.641501 28376 tablet_service.cc:1511] Processing CreateTablet for tablet a3c74d3ca66f42ad83ef54d9459b99fd (DEFAULT_TABLE table=default.Table_dabd3b5bd3ff4784b839d502baec0c6e [id=79633bfcdf584058b406f418caa25aea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.641801 28797 raft_consensus.cc:359] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.642040 28797 raft_consensus.cc:385] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.642136 28797 raft_consensus.cc:740] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.641945 28799 consensus_queue.cc:260] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.642205 28374 tablet_service.cc:1511] Processing CreateTablet for tablet 9d8f124ba79b46e098303b7b2705844d (DEFAULT_TABLE table=default.Table_6f1f355b18a34ac09304992c96d40765 [id=7bb817b09d154696977f3d68d9461d4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.642942 28524 tablet_service.cc:1511] Processing CreateTablet for tablet c871e7c8f843486da7ba40145c0fed19 (DEFAULT_TABLE table=default.Table_c6e2f443374a47d2a787049499a0b168 [id=e875c5c4a0004e8287426403b0b9c626]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.643158 28511 tablet_service.cc:1511] Processing CreateTablet for tablet 9d8f124ba79b46e098303b7b2705844d (DEFAULT_TABLE table=default.Table_6f1f355b18a34ac09304992c96d40765 [id=7bb817b09d154696977f3d68d9461d4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.643410 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c871e7c8f843486da7ba40145c0fed19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.643687 28511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d8f124ba79b46e098303b7b2705844d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.643700 28799 ts_tablet_manager.cc:1434] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260618 13:59:30.644016 28799 tablet_bootstrap.cc:492] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:30.644048 28516 tablet_service.cc:1511] Processing CreateTablet for tablet a3c74d3ca66f42ad83ef54d9459b99fd (DEFAULT_TABLE table=default.Table_dabd3b5bd3ff4784b839d502baec0c6e [id=79633bfcdf584058b406f418caa25aea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.644394 28516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3c74d3ca66f42ad83ef54d9459b99fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.646261 28799 tablet_bootstrap.cc:654] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.649073 28514 tablet_service.cc:1511] Processing CreateTablet for tablet 8fea86ca38a741238a59053151087f69 (DEFAULT_TABLE table=default.Table_7ae70108b1714e5ca374cc653bed7b33 [id=0cf9d7859b7d4d95bf769070c43003d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.649446 28514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fea86ca38a741238a59053151087f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.650136 29522 raft_consensus.cc:493] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.650271 29522 raft_consensus.cc:515] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.650956 29522 leader_election.cc:290] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:30.651916 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04dac931e62a448d99633878eb70b92f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:30.652482 28346 leader_election.cc:343] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.652788 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04dac931e62a448d99633878eb70b92f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:30.653147 28653 tablet_service.cc:1511] Processing CreateTablet for tablet c871e7c8f843486da7ba40145c0fed19 (DEFAULT_TABLE table=default.Table_c6e2f443374a47d2a787049499a0b168 [id=e875c5c4a0004e8287426403b0b9c626]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:30.653790 28346 leader_election.cc:343] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.653911 28346 leader_election.cc:304] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:30.654243 28797 consensus_queue.cc:260] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.654623 29522 raft_consensus.cc:2749] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:30.656173 28384 tablet_service.cc:1511] Processing CreateTablet for tablet 8fea86ca38a741238a59053151087f69 (DEFAULT_TABLE table=default.Table_7ae70108b1714e5ca374cc653bed7b33 [id=0cf9d7859b7d4d95bf769070c43003d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.656775 28406 raft_consensus.cc:2468] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:30.657238 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fea86ca38a741238a59053151087f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.657461 28376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3c74d3ca66f42ad83ef54d9459b99fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.657850 29572 ts_tablet_manager.cc:1929] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:30.657974 28374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d8f124ba79b46e098303b7b2705844d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.658032 29572 log.cc:1199] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/8b88ed7cf79c4929a8704f66691cbed4
I20260618 13:59:30.658563 29572 ts_tablet_manager.cc:1950] T 8b88ed7cf79c4929a8704f66691cbed4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:30.660305 28799 tablet_bootstrap.cc:492] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:30.660547 28799 ts_tablet_manager.cc:1403] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.003s
I20260618 13:59:30.661801 28373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c871e7c8f843486da7ba40145c0fed19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.654508 28657 tablet_service.cc:1511] Processing CreateTablet for tablet 9d8f124ba79b46e098303b7b2705844d (DEFAULT_TABLE table=default.Table_6f1f355b18a34ac09304992c96d40765 [id=7bb817b09d154696977f3d68d9461d4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.654523 28662 tablet_service.cc:1511] Processing CreateTablet for tablet a3c74d3ca66f42ad83ef54d9459b99fd (DEFAULT_TABLE table=default.Table_dabd3b5bd3ff4784b839d502baec0c6e [id=79633bfcdf584058b406f418caa25aea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.659081 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c871e7c8f843486da7ba40145c0fed19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.680365 28798 tablet_bootstrap.cc:492] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:30.680516 28798 ts_tablet_manager.cc:1403] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.046s user 0.003s sys 0.001s
I20260618 13:59:30.680536 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d8f124ba79b46e098303b7b2705844d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.681350 28798 raft_consensus.cc:359] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.681640 28798 raft_consensus.cc:385] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.681795 28798 raft_consensus.cc:740] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.682111 28798 consensus_queue.cc:260] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.682914 28549 raft_consensus.cc:3060] T 1829d1b69cdd4e748c5901c8e02cac2e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.686198 28799 raft_consensus.cc:359] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.686447 28799 raft_consensus.cc:385] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.686519 28799 raft_consensus.cc:740] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.686766 28799 consensus_queue.cc:260] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.687382 28799 ts_tablet_manager.cc:1434] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.027s user 0.000s sys 0.001s
I20260618 13:59:30.688398 28799 tablet_bootstrap.cc:492] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:30.691257 28799 tablet_bootstrap.cc:654] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.692270 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3c74d3ca66f42ad83ef54d9459b99fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.697772 28797 ts_tablet_manager.cc:1434] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.063s user 0.000s sys 0.003s
I20260618 13:59:30.700315 28798 ts_tablet_manager.cc:1434] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:30.700750 28798 tablet_bootstrap.cc:492] T c557afc3abfa4ef188bf248e89f49d87 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:30.703451 28798 tablet_bootstrap.cc:654] T c557afc3abfa4ef188bf248e89f49d87 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.707424 28799 tablet_bootstrap.cc:492] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:30.707619 28799 ts_tablet_manager.cc:1403] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.003s
I20260618 13:59:30.708447 28799 raft_consensus.cc:359] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.708660 28799 raft_consensus.cc:385] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.708753 28799 raft_consensus.cc:740] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.708989 28799 consensus_queue.cc:260] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.709901 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 8b88ed7cf79c4929a8704f66691cbed4 (table default.Table_1333d91d141a46acac33460026215195 [id=232810f09f9e41269d67ccd55527ab8a]) successfully deleted
I20260618 13:59:30.710253 28799 ts_tablet_manager.cc:1434] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:30.710558 28617 leader_election.cc:304] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:30.711280 28799 tablet_bootstrap.cc:492] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:30.711694 29572 tablet_replica.cc:333] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:30.711807 29400 raft_consensus.cc:2804] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:30.711905 29572 raft_consensus.cc:2243] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:30.712105 29572 raft_consensus.cc:2272] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:30.714061 28799 tablet_bootstrap.cc:654] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.719059 29400 raft_consensus.cc:697] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:30.719391 29400 consensus_queue.cc:237] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.720281 29607 consensus_queue.cc:1048] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
W20260618 13:59:30.720856 28617 consensus_peers.cc:597] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8b88ed7cf79c4929a8704f66691cbed4. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:30.721616 29607 consensus_queue.cc:1048] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.723122 29572 ts_tablet_manager.cc:1916] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:30.716543 28384 tablet_service.cc:1511] Processing CreateTablet for tablet 5bbb71449d434d54b007d6f0da0796d7 (DEFAULT_TABLE table=default.Table_571f6cccbfa3470a83b5542f128f2370 [id=9806bb85160042b095abdbcf04cf3748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.730918 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bbb71449d434d54b007d6f0da0796d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.734196 28524 tablet_service.cc:1511] Processing CreateTablet for tablet 5bbb71449d434d54b007d6f0da0796d7 (DEFAULT_TABLE table=default.Table_571f6cccbfa3470a83b5542f128f2370 [id=9806bb85160042b095abdbcf04cf3748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.734730 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bbb71449d434d54b007d6f0da0796d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.739408 28376 tablet_service.cc:1511] Processing CreateTablet for tablet d4539f767c054e9e8ebc1f3f56108e7e (DEFAULT_TABLE table=default.Table_d838306823ff4e22a68efe0a8cb751ff [id=bbfd0a6a47db4b1b8c534405b1068d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.739436 28516 tablet_service.cc:1511] Processing CreateTablet for tablet 3f5d4296952e48a48756e6f27140da87 (DEFAULT_TABLE table=default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.739902 28516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f5d4296952e48a48756e6f27140da87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.740280 28376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4539f767c054e9e8ebc1f3f56108e7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.744133 28654 tablet_service.cc:1511] Processing CreateTablet for tablet 5bbb71449d434d54b007d6f0da0796d7 (DEFAULT_TABLE table=default.Table_571f6cccbfa3470a83b5542f128f2370 [id=9806bb85160042b095abdbcf04cf3748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.745229 28799 tablet_bootstrap.cc:492] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:30.745478 28799 ts_tablet_manager.cc:1403] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.004s
I20260618 13:59:30.745707 28514 tablet_service.cc:1511] Processing CreateTablet for tablet d4539f767c054e9e8ebc1f3f56108e7e (DEFAULT_TABLE table=default.Table_d838306823ff4e22a68efe0a8cb751ff [id=bbfd0a6a47db4b1b8c534405b1068d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.746628 28514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4539f767c054e9e8ebc1f3f56108e7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.747660 28799 raft_consensus.cc:359] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.747874 28799 raft_consensus.cc:385] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.747946 28799 raft_consensus.cc:740] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.753778 28649 tablet_service.cc:1511] Processing CreateTablet for tablet 3f5d4296952e48a48756e6f27140da87 (DEFAULT_TABLE table=default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.753883 29489 raft_consensus.cc:493] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.754014 29489 raft_consensus.cc:515] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.754740 29489 leader_election.cc:290] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:30.754882 28663 tablet_service.cc:1511] Processing CreateTablet for tablet d4539f767c054e9e8ebc1f3f56108e7e (DEFAULT_TABLE table=default.Table_d838306823ff4e22a68efe0a8cb751ff [id=bbfd0a6a47db4b1b8c534405b1068d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.755579 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450c68da26404767b01c5e8da3f6286d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:30.756471 28346 leader_election.cc:343] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.758882 28797 tablet_bootstrap.cc:492] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:30.767055 28799 consensus_queue.cc:260] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.768149 28799 ts_tablet_manager.cc:1434] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.022s user 0.001s sys 0.001s
I20260618 13:59:30.769253 28384 tablet_service.cc:1511] Processing CreateTablet for tablet 3f5d4296952e48a48756e6f27140da87 (DEFAULT_TABLE table=default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.772962 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f5d4296952e48a48756e6f27140da87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.773154 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4539f767c054e9e8ebc1f3f56108e7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.775651 28649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f5d4296952e48a48756e6f27140da87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.779067 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bbb71449d434d54b007d6f0da0796d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.779171 29489 raft_consensus.cc:493] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.779336 29489 raft_consensus.cc:515] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.780050 29489 leader_election.cc:290] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:30.780864 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c881dd3cb66344969310956ad6e83cc0" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:30.781445 28346 leader_election.cc:343] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.782868 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c881dd3cb66344969310956ad6e83cc0" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:30.772379 28799 tablet_bootstrap.cc:492] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:30.783870 28346 leader_election.cc:343] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.784003 28346 leader_election.cc:304] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:30.784600 29489 raft_consensus.cc:2749] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:30.785497 29522 raft_consensus.cc:493] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.785645 29522 raft_consensus.cc:515] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.786335 29522 leader_election.cc:290] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:30.789088 29011 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 897bf6d6bbee4ad89c296325d94a3e53
I20260618 13:59:30.789601 29011 catalog_manager.cc:6784] ReplaceTablet: tablet 897bf6d6bbee4ad89c296325d94a3e53 deleted and replaced by tablet e196dd3c7fe14524a50152643ddbc2ad
I20260618 13:59:30.789740 29011 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 34) took 1056 ms. Trace:
I20260618 13:59:30.789844 29011 rpcz_store.cc:276] 0618 13:59:29.733092 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:30.396598 (+663506us) service_pool.cc:224] Handling call
0618 13:59:30.789685 (+393087us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:30.738285 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:30.738410 (+ 125us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:30.738459 (+ 49us) write_op.cc:435] Acquired schema lock
0618 13:59:30.738463 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:30.738594 (+ 131us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:30.738605 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:30.738611 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:30.738740 (+ 129us) tablet.cc:701] Row locks acquired
0618 13:59:30.738742 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:30.738791 (+ 49us) write_op.cc:270] Start()
0618 13:59:30.738831 (+ 40us) write_op.cc:276] Timestamp: P: 1781791170738750 usec, L: 0
0618 13:59:30.738835 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:30.739039 (+ 204us) log.cc:844] Serialized 625 byte log entry
0618 13:59:30.786632 (+ 47593us) op_driver.cc:464] REPLICATION: finished
0618 13:59:30.787431 (+ 799us) write_op.cc:301] APPLY: starting
0618 13:59:30.787441 (+ 10us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:30.787472 (+ 31us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:30.787499 (+ 27us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:30.787623 (+ 124us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:30.787682 (+ 59us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:30.787689 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:30.787759 (+ 70us) log.cc:844] Serialized 29 byte log entry
0618 13:59:30.787805 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:30.787820 (+ 15us) write_op.cc:454] Released schema lock
0618 13:59:30.787828 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:30.787892 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:30.787895 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":742,"apply.run_cpu_time_us":455,"apply.run_wall_time_us":455,"num_ops":2,"prepare.queue_time_us":116366,"prepare.run_cpu_time_us":934,"prepare.run_wall_time_us":8158,"raft.queue_time_us":1756,"raft.run_cpu_time_us":8,"raft.run_wall_time_us":9,"replication_time_us":47781}]]}
I20260618 13:59:30.786317 28799 tablet_bootstrap.cc:654] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.799691 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c75278db2234b11bf95f1cf5cc76edd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:30.800359 28346 leader_election.cc:343] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.800972 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet 897bf6d6bbee4ad89c296325d94a3e53 with delete_type TABLET_DATA_DELETED (replaced by tablet e196dd3c7fe14524a50152643ddbc2ad) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:30.802214 28655 tablet_service.cc:1511] Processing CreateTablet for tablet 8fea86ca38a741238a59053151087f69 (DEFAULT_TABLE table=default.Table_7ae70108b1714e5ca374cc653bed7b33 [id=0cf9d7859b7d4d95bf769070c43003d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:30.803504 28799 tablet_bootstrap.cc:492] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:30.803700 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c75278db2234b11bf95f1cf5cc76edd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:30.803798 28799 ts_tablet_manager.cc:1403] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.003s
W20260618 13:59:30.804775 28346 leader_election.cc:343] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.806794 28346 leader_election.cc:304] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:30.807516 29489 raft_consensus.cc:2749] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:30.804778 28799 raft_consensus.cc:359] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.808123 28799 raft_consensus.cc:385] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.808303 28799 raft_consensus.cc:740] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.808737 28799 consensus_queue.cc:260] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.810441 28799 ts_tablet_manager.cc:1434] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260618 13:59:30.810931 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fea86ca38a741238a59053151087f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:30.806520 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450c68da26404767b01c5e8da3f6286d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:30.821666 28346 leader_election.cc:343] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.821838 28346 leader_election.cc:304] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:30.822494 29489 raft_consensus.cc:2749] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:30.823347 28799 tablet_bootstrap.cc:492] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:30.827454 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet 897bf6d6bbee4ad89c296325d94a3e53 with delete_type TABLET_DATA_DELETED (replaced by tablet e196dd3c7fe14524a50152643ddbc2ad) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:30.827395 28799 tablet_bootstrap.cc:654] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.844511 28797 tablet_bootstrap.cc:654] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.852612 28798 tablet_bootstrap.cc:492] T c557afc3abfa4ef188bf248e89f49d87 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:30.852988 29572 ts_tablet_manager.cc:1929] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:30.853166 29572 log.cc:1199] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/233040a16b0d4b61b977ca57c1afec2d
I20260618 13:59:30.853663 29489 raft_consensus.cc:493] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.853713 29572 ts_tablet_manager.cc:1950] T 233040a16b0d4b61b977ca57c1afec2d P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:30.853853 29489 raft_consensus.cc:515] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.854591 29489 leader_election.cc:290] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:30.855392 29572 tablet_replica.cc:333] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:30.855559 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 233040a16b0d4b61b977ca57c1afec2d (table default.Table_7e5df99a84c84d66b548875d6d934656 [id=2332167cba774a0795c11d97505049c9]) successfully deleted
I20260618 13:59:30.855648 29572 raft_consensus.cc:2243] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:30.855849 29572 raft_consensus.cc:2272] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:30.855897 28988 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet f3ba9186b3ff47d5a2b158fef619686e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f3ba9186b3ff47d5a2b158fef619686e already in progress: creating tablet
W20260618 13:59:30.856602 28988 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet ca2ee746c5024cfc9b31c470f9616e7d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ca2ee746c5024cfc9b31c470f9616e7d already in progress: creating tablet
I20260618 13:59:30.855615 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce7bf74b6d24d0bb43808b8a62cf1b7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:30.857308 28346 leader_election.cc:343] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.857689 29572 ts_tablet_manager.cc:1916] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:30.857756 28798 ts_tablet_manager.cc:1403] T c557afc3abfa4ef188bf248e89f49d87 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.157s user 0.003s sys 0.003s
I20260618 13:59:30.861918 28798 raft_consensus.cc:359] T c557afc3abfa4ef188bf248e89f49d87 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.862265 28798 raft_consensus.cc:385] T c557afc3abfa4ef188bf248e89f49d87 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.862534 28798 raft_consensus.cc:740] T c557afc3abfa4ef188bf248e89f49d87 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.862928 28798 consensus_queue.cc:260] T c557afc3abfa4ef188bf248e89f49d87 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.865088 28798 ts_tablet_manager.cc:1434] T c557afc3abfa4ef188bf248e89f49d87 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260618 13:59:30.867132 29400 raft_consensus.cc:493] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.867357 29400 raft_consensus.cc:515] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.868903 28798 tablet_bootstrap.cc:492] T aa8cf927bed9437086efd8e0f21ba26a P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:30.867690 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce7bf74b6d24d0bb43808b8a62cf1b7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:30.870162 28346 leader_election.cc:343] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.870319 28346 leader_election.cc:304] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:30.870489 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece7909d056c4ba8a5dca6423f0c1781" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:30.870777 29489 raft_consensus.cc:2749] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:30.870795 28404 raft_consensus.cc:2468] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:30.873374 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:30.875931 28798 tablet_bootstrap.cc:654] T aa8cf927bed9437086efd8e0f21ba26a P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:30.880630 28983 consensus_peers.cc:597] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 -> Peer 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083): Couldn't send request to peer 7637285570574a6dbcef1de6f84ecac0. Status: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260618 13:59:30.883759 29024 catalog_manager.cc:5696] T a1a06078974e4bfbbc4b949802a39bf1 P 97859748af2c41879ba2a668dda999d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } }
I20260618 13:59:30.884348 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet a1a06078974e4bfbbc4b949802a39bf1 (table default.Table_f958c4ce58c34bfda98454664e202af2 [id=7b747ef3b2494ca78c499c5bb75648d4]). Expected version 2 got 0
I20260618 13:59:30.884609 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet ece7909d056c4ba8a5dca6423f0c1781 (table default.Table_26e7d8c4f91144268a2a43e83992667d [id=803d5220ce1a4dcfaf27d93c574be2ac]). Expected version 1 got 0
I20260618 13:59:30.881666 29017 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_dc44ae6b83d844b6a16ce10ba6dce0ad" } new_table_name: "default.Table_5c88f7a919ad4d5da30ed365c5533883" modify_external_catalogs: true
I20260618 13:59:30.885000 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet dc36fbea853a4d4e9f61909200239aed (table default.Table_dbc6162c06f74e71b792bd1f7b2388fe [id=49576dabdecf41aca1b15ff3fc0c113c]). Expected version 6 got 0
I20260618 13:59:30.885572 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:30.885874 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 15180aefba0544329daee74308ef5010 (table default.Table_97b6bb39b3bf4612be391b9dd78d1646 [id=42a2a24590834abc9537669aaf223a7c]). Expected version 1 got 0
I20260618 13:59:30.886036 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:30.886315 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9c17e76ae9b44207998a43e3576908c4 (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]). Expected version 2 got 0
I20260618 13:59:30.887487 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 98b42d23832545ae81a7cdff19e4689f (table default.Table_c99584ecc88948a097b39c22d68b3938 [id=7c39b3c432e944bca05f7b2070a79ff9]). Expected version 1 got 0
I20260618 13:59:30.887776 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:30.886591 28617 leader_election.cc:304] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:30.888865 29400 leader_election.cc:290] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:30.889051 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:30.889189 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 897bf6d6bbee4ad89c296325d94a3e53 with delete_type TABLET_DATA_DELETED (replaced by tablet e196dd3c7fe14524a50152643ddbc2ad) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:30.891698 29400 raft_consensus.cc:2804] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:30.891834 29400 raft_consensus.cc:493] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:30.892637 29400 raft_consensus.cc:3060] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.902082 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9b75ece59d924e3db49e002bf0faa89c (table default.Table_031e1a03030b45c99c1270d612008f91 [id=387d3ff1762749efad926181fe4c746a]). Expected version 1 got 0
I20260618 13:59:30.902946 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:30.903726 28798 tablet_bootstrap.cc:492] T aa8cf927bed9437086efd8e0f21ba26a P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:30.903937 28798 ts_tablet_manager.cc:1403] T aa8cf927bed9437086efd8e0f21ba26a P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.005s
I20260618 13:59:30.904768 28798 raft_consensus.cc:359] T aa8cf927bed9437086efd8e0f21ba26a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:30.905040 28798 raft_consensus.cc:385] T aa8cf927bed9437086efd8e0f21ba26a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.905293 28798 raft_consensus.cc:740] T aa8cf927bed9437086efd8e0f21ba26a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.905617 28798 consensus_queue.cc:260] T aa8cf927bed9437086efd8e0f21ba26a P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:30.907125 28798 ts_tablet_manager.cc:1434] T aa8cf927bed9437086efd8e0f21ba26a P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260618 13:59:30.907495 28797 tablet_bootstrap.cc:492] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:30.907682 28797 ts_tablet_manager.cc:1403] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.209s user 0.002s sys 0.007s
I20260618 13:59:30.908308 28798 tablet_bootstrap.cc:492] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:30.908469 28797 raft_consensus.cc:359] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.908659 28797 raft_consensus.cc:385] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.908759 28797 raft_consensus.cc:740] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.908985 28797 consensus_queue.cc:260] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.909529 28797 ts_tablet_manager.cc:1434] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:30.910230 28797 tablet_bootstrap.cc:492] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:30.910491 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:30.911722 28798 tablet_bootstrap.cc:654] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.912082 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:30.912844 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 512a9e63ba324a30b8d8c37e585ebdf8 (table default.Table_c70bc1c98b3b49d7aa3080904667732d [id=901874cb80e74ba589679bb7b95c8fe4]). Expected version 1 got 0
I20260618 13:59:30.912453 28549 raft_consensus.cc:3060] T e86aeb21427e4859a5266f4451469b25 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.913219 28547 raft_consensus.cc:3060] T c557afc3abfa4ef188bf248e89f49d87 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.914764 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 6b8f273def3e4b7caa4d624cf87a106c (table default.Table_5159a242b12044fc9a0b3e535ea40050 [id=cfb4ca8c4e884ccc8602be52d0b39100]). Expected version 4 got 0
I20260618 13:59:30.915242 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet f278d0acbb0c4a0ca61640bc142e6335 (table default.Table_8783a8d8c526428281436b62b2581a33 [id=0d83fc68255c458587390c58b12e00d7]). Expected version 1 got 0
I20260618 13:59:30.915556 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 39b06ae7444a48dbbec9e096fd7e4052 (table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15]). Expected version 2 got 0
I20260618 13:59:30.915805 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:30.916422 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:30.916673 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet cea4bc13eabc4dd98b4f96723498014c (table default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c]). Expected version 1 got 0
I20260618 13:59:30.917837 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:30.918282 29024 catalog_manager.cc:5696] T 1f81428f804a485c967026c4527fd250 P 97859748af2c41879ba2a668dda999d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } }
I20260618 13:59:30.921509 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 1f81428f804a485c967026c4527fd250 (table default.Table_706eec6806a94d3f8d5b60708a0c1cda [id=b817db93d50144d7b7c238550eb53b10]). Expected version 1 got 0
I20260618 13:59:30.921762 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:30.922355 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:30.922641 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:30.924656 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:30.924945 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_efb2b8e451084b7ba0de07e0a6e538c2 [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 1 got 0
I20260618 13:59:30.925242 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:30.925570 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_12d6cc95071949c6a7a297cd08b5f898 [id=840c933184144aebb3659d939f679f00]). Expected version 1 got 0
I20260618 13:59:30.926049 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:30.931337 28797 tablet_bootstrap.cc:654] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.934653 28548 raft_consensus.cc:3060] T aa8cf927bed9437086efd8e0f21ba26a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.935503 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece7909d056c4ba8a5dca6423f0c1781" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:30.935874 29640 raft_consensus.cc:493] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.936118 29640 raft_consensus.cc:515] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.936919 29640 leader_election.cc:290] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:30.935745 28618 consensus_peers.cc:597] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:30.937845 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827cc0987bd541759e334fb620f04ed5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:30.938035 28407 raft_consensus.cc:2468] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:30.938182 28618 leader_election.cc:343] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.938772 28617 leader_election.cc:304] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:30.939074 29640 consensus_queue.cc:1048] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.939512 29640 raft_consensus.cc:2804] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:30.939601 29640 raft_consensus.cc:493] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:30.939713 29640 raft_consensus.cc:3060] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.941893 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827cc0987bd541759e334fb620f04ed5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:30.948928 29413 consensus_queue.cc:1048] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:30.951898 28618 leader_election.cc:343] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.953327 28798 tablet_bootstrap.cc:492] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:30.953595 28798 ts_tablet_manager.cc:1403] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.046s user 0.001s sys 0.005s
I20260618 13:59:30.954440 28798 raft_consensus.cc:359] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.954806 28798 raft_consensus.cc:385] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:30.955070 28798 raft_consensus.cc:740] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:30.955382 28798 consensus_queue.cc:260] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.955969 28798 ts_tablet_manager.cc:1434] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 13:59:30.956358 28798 tablet_bootstrap.cc:492] T 897bf6d6bbee4ad89c296325d94a3e53 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:30.957285 29570 ts_tablet_manager.cc:1929] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:30.957451 29570 log.cc:1199] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/edc72a06ee8e4420b07a17d2feb3fcc2
I20260618 13:59:30.957826 29570 ts_tablet_manager.cc:1950] T edc72a06ee8e4420b07a17d2feb3fcc2 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:30.962381 29572 ts_tablet_manager.cc:1929] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:30.962659 29572 log.cc:1199] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/897bf6d6bbee4ad89c296325d94a3e53
I20260618 13:59:30.963153 29572 ts_tablet_manager.cc:1950] T 897bf6d6bbee4ad89c296325d94a3e53 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:30.966570 28798 tablet_bootstrap.cc:654] T 897bf6d6bbee4ad89c296325d94a3e53 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:30.962150 29400 raft_consensus.cc:515] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:30.967393 29400 leader_election.cc:290] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:30.963114 29566 ts_tablet_manager.cc:1929] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:30.968062 29566 log.cc:1199] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/edc72a06ee8e4420b07a17d2feb3fcc2
I20260618 13:59:30.968099 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece7909d056c4ba8a5dca6423f0c1781" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:30.968317 28404 raft_consensus.cc:3060] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:30.968510 29566 ts_tablet_manager.cc:1950] T edc72a06ee8e4420b07a17d2feb3fcc2 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
W20260618 13:59:30.972070 28618 consensus_peers.cc:597] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:30.972163 29400 consensus_queue.cc:1048] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:30.972080 29607 consensus_queue.cc:1048] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260618 13:59:30.972713 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece7909d056c4ba8a5dca6423f0c1781" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:30.973896 28618 leader_election.cc:343] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:30.976171 29570 tablet_replica.cc:333] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:30.976384 29570 raft_consensus.cc:2243] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:30.976773 29570 raft_consensus.cc:2272] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:30.977321 29570 ts_tablet_manager.cc:1916] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:30.984648 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet edc72a06ee8e4420b07a17d2feb3fcc2 (table default.Table_cf80fd5bbdde4cfb855c8fbe860a15f3 [id=9d63ad7327a44124af817b465a9ae0da]) successfully deleted
I20260618 13:59:30.992605 29489 raft_consensus.cc:493] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:30.992880 29489 raft_consensus.cc:515] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:30.993726 29489 leader_election.cc:290] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:30.995213 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5a334d18ba4c06bb358fa86d8af80d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:30.995739 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5a334d18ba4c06bb358fa86d8af80d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:30.996389 28346 leader_election.cc:343] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:31.002622 28346 leader_election.cc:343] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.002784 28346 leader_election.cc:304] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:31.003291 29489 raft_consensus.cc:2749] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:31.003937 29400 consensus_queue.cc:1048] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:31.013543 28618 consensus_peers.cc:597] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:31.013772 28544 raft_consensus.cc:3060] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.016214 28798 tablet_bootstrap.cc:492] T 897bf6d6bbee4ad89c296325d94a3e53 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:31.017017 28798 ts_tablet_manager.cc:1403] T 897bf6d6bbee4ad89c296325d94a3e53 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.061s user 0.000s sys 0.004s
I20260618 13:59:31.022774 28798 raft_consensus.cc:359] T 897bf6d6bbee4ad89c296325d94a3e53 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.034557 28798 raft_consensus.cc:385] T 897bf6d6bbee4ad89c296325d94a3e53 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.034778 28798 raft_consensus.cc:740] T 897bf6d6bbee4ad89c296325d94a3e53 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.035143 28798 consensus_queue.cc:260] T 897bf6d6bbee4ad89c296325d94a3e53 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.040354 28798 ts_tablet_manager.cc:1434] T 897bf6d6bbee4ad89c296325d94a3e53 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.022s user 0.002s sys 0.000s
I20260618 13:59:31.040795 28798 tablet_bootstrap.cc:492] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:31.041862 29489 raft_consensus.cc:493] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.042044 29489 raft_consensus.cc:515] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.043080 29489 leader_election.cc:290] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.043999 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a10e83d26b704201b95e7c0c44a1a4f5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:31.044567 28346 leader_election.cc:343] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.045635 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a10e83d26b704201b95e7c0c44a1a4f5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:31.045795 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 897bf6d6bbee4ad89c296325d94a3e53 (table default.Table_35260f90f6814fa3915b4280cff96d1b [id=02ffed3d852845d6845fb5176f036fce]) successfully deleted
W20260618 13:59:31.046380 28346 leader_election.cc:343] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.046557 28346 leader_election.cc:304] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:31.047232 29489 raft_consensus.cc:2749] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:31.052264 28798 tablet_bootstrap.cc:654] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.053612 28799 tablet_bootstrap.cc:492] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:31.053784 28799 ts_tablet_manager.cc:1403] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.231s user 0.006s sys 0.020s
I20260618 13:59:31.054642 28799 raft_consensus.cc:359] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.054862 28799 raft_consensus.cc:385] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.054947 28799 raft_consensus.cc:740] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.055766 28404 raft_consensus.cc:2468] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:31.055200 28799 consensus_queue.cc:260] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.062325 28799 ts_tablet_manager.cc:1434] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260618 13:59:31.062674 28799 tablet_bootstrap.cc:492] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:31.066802 28798 tablet_bootstrap.cc:492] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:31.067054 28798 ts_tablet_manager.cc:1403] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.002s
I20260618 13:59:31.067075 28799 tablet_bootstrap.cc:654] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.068011 28798 raft_consensus.cc:359] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.068338 28798 raft_consensus.cc:385] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.068468 28798 raft_consensus.cc:740] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.068967 28798 consensus_queue.cc:260] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.070058 28798 ts_tablet_manager.cc:1434] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:31.070442 28798 tablet_bootstrap.cc:492] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:31.072471 28798 tablet_bootstrap.cc:654] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.077669 29221 raft_consensus.cc:493] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.077811 29221 raft_consensus.cc:515] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.078408 29221 leader_election.cc:290] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:31.079180 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d667ea5f029f46d89fcb3be03069b97b" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:31.085630 28799 tablet_bootstrap.cc:492] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:31.085793 28799 ts_tablet_manager.cc:1403] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.023s user 0.008s sys 0.002s
I20260618 13:59:31.086666 28799 raft_consensus.cc:359] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.086990 28799 raft_consensus.cc:385] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.087157 28799 raft_consensus.cc:740] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.087482 28799 consensus_queue.cc:260] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.090899 28799 ts_tablet_manager.cc:1434] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260618 13:59:31.091747 28799 tablet_bootstrap.cc:492] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:31.093597 28799 tablet_bootstrap.cc:654] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.094538 29644 consensus_queue.cc:1048] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260618 13:59:31.094591 29640 raft_consensus.cc:515] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.095639 28617 leader_election.cc:304] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:31.097579 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet edc72a06ee8e4420b07a17d2feb3fcc2 (table default.Table_cf80fd5bbdde4cfb855c8fbe860a15f3 [id=9d63ad7327a44124af817b465a9ae0da]) successfully deleted
I20260618 13:59:31.098718 29566 tablet_replica.cc:333] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:31.104710 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827cc0987bd541759e334fb620f04ed5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:31.104910 28404 raft_consensus.cc:3060] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.111418 29566 raft_consensus.cc:2243] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:31.112499 29644 raft_consensus.cc:2804] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
W20260618 13:59:31.112831 28618 consensus_peers.cc:597] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:31.113678 29640 leader_election.cc:290] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.114379 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d667ea5f029f46d89fcb3be03069b97b" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:31.115300 28484 leader_election.cc:304] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:31.115861 29221 raft_consensus.cc:2749] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:31.116240 29644 raft_consensus.cc:697] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:31.117617 28548 raft_consensus.cc:3060] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.120846 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827cc0987bd541759e334fb620f04ed5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:31.121780 29566 raft_consensus.cc:2272] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:31.123909 29566 ts_tablet_manager.cc:1916] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:31.123868 28514 tablet_service.cc:1511] Processing CreateTablet for tablet e3a3395aa90e426eb48a3920a8c45ba8 (DEFAULT_TABLE table=default.Table_df7ccb8b212342b5ad9837bf98607ec9 [id=daa133fc6f89489f91ef6cadcf30fbdb]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:31.123881 28618 leader_election.cc:343] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.125377 29644 consensus_queue.cc:237] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.136083 28662 tablet_service.cc:1511] Processing CreateTablet for tablet e3a3395aa90e426eb48a3920a8c45ba8 (DEFAULT_TABLE table=default.Table_df7ccb8b212342b5ad9837bf98607ec9 [id=daa133fc6f89489f91ef6cadcf30fbdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.136560 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3a3395aa90e426eb48a3920a8c45ba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.136768 28384 tablet_service.cc:1511] Processing CreateTablet for tablet e3a3395aa90e426eb48a3920a8c45ba8 (DEFAULT_TABLE table=default.Table_df7ccb8b212342b5ad9837bf98607ec9 [id=daa133fc6f89489f91ef6cadcf30fbdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.137257 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3a3395aa90e426eb48a3920a8c45ba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.140411 28524 tablet_service.cc:1511] Processing CreateTablet for tablet db62b3a1e37b447c9ea3fbd65c0cf1e7 (DEFAULT_TABLE table=default.Table_f645609880ff40afb013281b28341bca [id=08f4eb14a0af45c8981be232c11bfb40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.142149 28376 tablet_service.cc:1511] Processing CreateTablet for tablet db62b3a1e37b447c9ea3fbd65c0cf1e7 (DEFAULT_TABLE table=default.Table_f645609880ff40afb013281b28341bca [id=08f4eb14a0af45c8981be232c11bfb40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.142529 28376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db62b3a1e37b447c9ea3fbd65c0cf1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.142854 28516 tablet_service.cc:1511] Processing CreateTablet for tablet 6c4a31a4f202473d97b17b5b5dfdb615 (DEFAULT_TABLE table=default.Table_152a483501b54a67bbdc0cae4959d452 [id=daf2b982b88e4021b327a0e52cf5316c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.142835 28655 tablet_service.cc:1511] Processing CreateTablet for tablet db62b3a1e37b447c9ea3fbd65c0cf1e7 (DEFAULT_TABLE table=default.Table_f645609880ff40afb013281b28341bca [id=08f4eb14a0af45c8981be232c11bfb40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.143210 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db62b3a1e37b447c9ea3fbd65c0cf1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.146616 28373 tablet_service.cc:1511] Processing CreateTablet for tablet 6c4a31a4f202473d97b17b5b5dfdb615 (DEFAULT_TABLE table=default.Table_152a483501b54a67bbdc0cae4959d452 [id=daf2b982b88e4021b327a0e52cf5316c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.147073 28373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c4a31a4f202473d97b17b5b5dfdb615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.148403 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 1f81428f804a485c967026c4527fd250 (table default.Table_706eec6806a94d3f8d5b60708a0c1cda [id=b817db93d50144d7b7c238550eb53b10]). Expected version 1 got 0
I20260618 13:59:31.148613 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9b75ece59d924e3db49e002bf0faa89c (table default.Table_031e1a03030b45c99c1270d612008f91 [id=387d3ff1762749efad926181fe4c746a]). Expected version 1 got 0
I20260618 13:59:31.148905 29025 catalog_manager.cc:5696] T 1829d1b69cdd4e748c5901c8e02cac2e P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:31.149410 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:31.149580 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:31.149811 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 512a9e63ba324a30b8d8c37e585ebdf8 (table default.Table_c70bc1c98b3b49d7aa3080904667732d [id=901874cb80e74ba589679bb7b95c8fe4]). Expected version 1 got 0
I20260618 13:59:31.150336 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet dc36fbea853a4d4e9f61909200239aed (table default.Table_dbc6162c06f74e71b792bd1f7b2388fe [id=49576dabdecf41aca1b15ff3fc0c113c]). Expected version 6 got 0
I20260618 13:59:31.151584 28511 tablet_service.cc:1511] Processing CreateTablet for tablet e3d8616846c649b6a969b971db0d03db (DEFAULT_TABLE table=default.Table_473a0c23203b404ea8e5ee0aed201e7b [id=3656a2f8d6cc4892a071908ae55efd3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.152590 28653 tablet_service.cc:1511] Processing CreateTablet for tablet 6c4a31a4f202473d97b17b5b5dfdb615 (DEFAULT_TABLE table=default.Table_152a483501b54a67bbdc0cae4959d452 [id=daf2b982b88e4021b327a0e52cf5316c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.152983 28657 tablet_service.cc:1511] Processing CreateTablet for tablet e3d8616846c649b6a969b971db0d03db (DEFAULT_TABLE table=default.Table_473a0c23203b404ea8e5ee0aed201e7b [id=3656a2f8d6cc4892a071908ae55efd3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.153571 28374 tablet_service.cc:1511] Processing CreateTablet for tablet e3d8616846c649b6a969b971db0d03db (DEFAULT_TABLE table=default.Table_473a0c23203b404ea8e5ee0aed201e7b [id=3656a2f8d6cc4892a071908ae55efd3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.153055 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c4a31a4f202473d97b17b5b5dfdb615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.153847 28517 tablet_service.cc:1511] Processing CreateTablet for tablet 5115a31a38db48c1a6474917aff97c8e (DEFAULT_TABLE table=default.Table_94aa74339cf547a080b6ede6a36b4bed [id=9006108b57df43d8b559708d300e79ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.154577 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3d8616846c649b6a969b971db0d03db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.155990 28522 tablet_service.cc:1511] Processing CreateTablet for tablet d8a71e334ed1400d97c965b49b677a81 (DEFAULT_TABLE table=default.Table_76b9ce534bf440919c76e1c8005ecc57 [id=445f1af32571418e979bbbbff739c798]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.158018 28529 tablet_service.cc:1511] Processing CreateTablet for tablet 10e9a417eecf4d33ac3bd9bd6325f447 (DEFAULT_TABLE table=default.Table_819e5138e3c7415a80631f3cbb6f8b81 [id=0de5461053eb4517b15d6b91875fc04d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.159273 29570 ts_tablet_manager.cc:1929] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:31.159389 29570 log.cc:1199] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/4a01dfb4d59549399c94640f16045555
I20260618 13:59:31.159787 29570 ts_tablet_manager.cc:1950] T 4a01dfb4d59549399c94640f16045555 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:31.160559 28514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3a3395aa90e426eb48a3920a8c45ba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.160894 28798 tablet_bootstrap.cc:492] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:31.161053 28798 ts_tablet_manager.cc:1403] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.091s user 0.005s sys 0.000s
I20260618 13:59:31.161127 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db62b3a1e37b447c9ea3fbd65c0cf1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.161571 28516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c4a31a4f202473d97b17b5b5dfdb615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.161917 28511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3d8616846c649b6a969b971db0d03db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.162309 28517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5115a31a38db48c1a6474917aff97c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.162251 28798 raft_consensus.cc:359] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.162487 28798 raft_consensus.cc:385] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.162586 28798 raft_consensus.cc:740] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.162712 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8a71e334ed1400d97c965b49b677a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.162879 28798 consensus_queue.cc:260] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.163115 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10e9a417eecf4d33ac3bd9bd6325f447. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.164207 28798 ts_tablet_manager.cc:1434] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260618 13:59:31.164773 28654 tablet_service.cc:1511] Processing CreateTablet for tablet 10e9a417eecf4d33ac3bd9bd6325f447 (DEFAULT_TABLE table=default.Table_819e5138e3c7415a80631f3cbb6f8b81 [id=0de5461053eb4517b15d6b91875fc04d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.164980 28663 tablet_service.cc:1511] Processing CreateTablet for tablet d8a71e334ed1400d97c965b49b677a81 (DEFAULT_TABLE table=default.Table_76b9ce534bf440919c76e1c8005ecc57 [id=445f1af32571418e979bbbbff739c798]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.165184 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10e9a417eecf4d33ac3bd9bd6325f447. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.165341 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8a71e334ed1400d97c965b49b677a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.176174 28653 tablet_service.cc:1511] Processing CreateTablet for tablet 5115a31a38db48c1a6474917aff97c8e (DEFAULT_TABLE table=default.Table_94aa74339cf547a080b6ede6a36b4bed [id=9006108b57df43d8b559708d300e79ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.169515 29570 tablet_replica.cc:333] T 897bf6d6bbee4ad89c296325d94a3e53 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:31.176947 29570 raft_consensus.cc:2243] T 897bf6d6bbee4ad89c296325d94a3e53 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:31.177136 29570 raft_consensus.cc:2272] T 897bf6d6bbee4ad89c296325d94a3e53 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:31.175992 29221 raft_consensus.cc:493] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.177515 29221 raft_consensus.cc:515] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.178023 29570 ts_tablet_manager.cc:1916] T 897bf6d6bbee4ad89c296325d94a3e53 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:31.178241 29221 leader_election.cc:290] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:31.179149 28798 tablet_bootstrap.cc:492] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:31.179566 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46bfe8cc2078414d84e65e22ed4ef8a2" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:31.179600 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 98b42d23832545ae81a7cdff19e4689f (table default.Table_c99584ecc88948a097b39c22d68b3938 [id=7c39b3c432e944bca05f7b2070a79ff9]). Expected version 1 got 0
I20260618 13:59:31.179841 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:31.180042 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:31.182103 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 15180aefba0544329daee74308ef5010 (table default.Table_97b6bb39b3bf4612be391b9dd78d1646 [id=42a2a24590834abc9537669aaf223a7c]). Expected version 1 got 0
I20260618 13:59:31.184937 29221 raft_consensus.cc:493] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.185104 29221 raft_consensus.cc:515] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:31.185525 28514 tablet_service.cc:1511] Processing CreateTablet for tablet a8da96fc621c47d4a444b5a39259f85d (DEFAULT_TABLE table=default.Table_a7ff69763fdb4ff3b26b180bf05907d9 [id=e39e2c5718794a748361d7b453c65404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.185786 29221 leader_election.cc:290] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:31.184185 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 4a01dfb4d59549399c94640f16045555 (table default.Table_7700ee6bae434a01b621ad02088cc5f0 [id=fe7eabed8da6400bbb721520c6182f8c]) successfully deleted
W20260618 13:59:31.186269 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet f3ba9186b3ff47d5a2b158fef619686e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f3ba9186b3ff47d5a2b158fef619686e already in progress: creating tablet
W20260618 13:59:31.186502 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet ca2ee746c5024cfc9b31c470f9616e7d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ca2ee746c5024cfc9b31c470f9616e7d already in progress: creating tablet
W20260618 13:59:31.186808 28484 leader_election.cc:343] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: e025b64380314546af8a1ab90c817825
I20260618 13:59:31.187335 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:31.188555 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9c17e76ae9b44207998a43e3576908c4 (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]). Expected version 2 got 0
I20260618 13:59:31.189893 28519 tablet_service.cc:1511] Processing CreateTablet for tablet fea86899d2c047d997cebe80551e83c9 (DEFAULT_TABLE table=default.Table_539ac216b6f4478eb994218d7d7980a4 [id=baa3214376034e9cae926def4c46b91e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.195014 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:31.195567 29640 raft_consensus.cc:493] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.195755 29640 raft_consensus.cc:515] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:31.195928 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:31.196135 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet a1a06078974e4bfbbc4b949802a39bf1 (table default.Table_f958c4ce58c34bfda98454664e202af2 [id=7b747ef3b2494ca78c499c5bb75648d4]). Expected version 2 got 0
I20260618 13:59:31.196280 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet ece7909d056c4ba8a5dca6423f0c1781 (table default.Table_26e7d8c4f91144268a2a43e83992667d [id=803d5220ce1a4dcfaf27d93c574be2ac]). Expected version 1 got 0
I20260618 13:59:31.196439 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:31.196753 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:31.196902 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 6b8f273def3e4b7caa4d624cf87a106c (table default.Table_5159a242b12044fc9a0b3e535ea40050 [id=cfb4ca8c4e884ccc8602be52d0b39100]). Expected version 4 got 0
I20260618 13:59:31.196884 29640 leader_election.cc:290] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.198840 28513 tablet_service.cc:1511] Processing CreateTablet for tablet 2e493f647a344cd581515a77230d765f (DEFAULT_TABLE table=default.Table_7699960fdee24242bf51bcfcbd6f68c2 [id=3b9ab803536344d3a59ff77b2b58ae31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.201179 28374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3d8616846c649b6a969b971db0d03db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.201700 28404 raft_consensus.cc:2468] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:31.201979 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a42aa779ae452e9420d730bf8d7d55" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:31.202172 28407 raft_consensus.cc:2468] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:31.202407 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_efb2b8e451084b7ba0de07e0a6e538c2 [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 1 got 0
I20260618 13:59:31.202688 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:31.202834 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:31.203938 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:31.204030 29489 raft_consensus.cc:493] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.204109 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:31.204846 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a42aa779ae452e9420d730bf8d7d55" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:31.205349 28618 leader_election.cc:343] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.205477 28528 tablet_service.cc:1511] Processing CreateTablet for tablet a662bb37fb7d4df49e61e524808b67a8 (DEFAULT_TABLE table=default.Table_0a85d79af65f48a3803bfd1248593a11 [id=2f4d62be772a421ab777fd23be58b719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.205708 28617 leader_election.cc:304] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:31.205778 28655 tablet_service.cc:1511] Processing CreateTablet for tablet fea86899d2c047d997cebe80551e83c9 (DEFAULT_TABLE table=default.Table_539ac216b6f4478eb994218d7d7980a4 [id=baa3214376034e9cae926def4c46b91e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.206125 28617 leader_election.cc:304] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:31.206650 28649 tablet_service.cc:1511] Processing CreateTablet for tablet a662bb37fb7d4df49e61e524808b67a8 (DEFAULT_TABLE table=default.Table_0a85d79af65f48a3803bfd1248593a11 [id=2f4d62be772a421ab777fd23be58b719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.206907 29413 raft_consensus.cc:2804] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:31.207051 29413 raft_consensus.cc:697] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:31.207561 29402 raft_consensus.cc:2804] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:31.207644 29402 raft_consensus.cc:493] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:31.207785 29402 raft_consensus.cc:3060] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.207754 29413 consensus_queue.cc:237] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.208186 28657 tablet_service.cc:1511] Processing CreateTablet for tablet a8da96fc621c47d4a444b5a39259f85d (DEFAULT_TABLE table=default.Table_a7ff69763fdb4ff3b26b180bf05907d9 [id=e39e2c5718794a748361d7b453c65404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.208511 28797 tablet_bootstrap.cc:492] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:31.208688 28797 ts_tablet_manager.cc:1403] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.298s user 0.014s sys 0.010s
W20260618 13:59:31.209622 29672 log.cc:927] Time spent T aa8cf927bed9437086efd8e0f21ba26a P 97859748af2c41879ba2a668dda999d2: Append to log took a long time: real 0.086s user 0.000s sys 0.000s
I20260618 13:59:31.209527 28797 raft_consensus.cc:359] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.209740 28797 raft_consensus.cc:385] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.210888 28797 raft_consensus.cc:740] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.211110 28797 consensus_queue.cc:260] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.211380 28799 tablet_bootstrap.cc:492] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:31.211619 28797 ts_tablet_manager.cc:1434] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260618 13:59:31.211851 28797 tablet_bootstrap.cc:492] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:31.213218 28797 tablet_bootstrap.cc:654] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.214802 29489 raft_consensus.cc:515] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.215397 29489 leader_election.cc:290] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.215441 28797 tablet_bootstrap.cc:492] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:31.215576 28797 ts_tablet_manager.cc:1403] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260618 13:59:31.216174 28525 tablet_service.cc:1511] Processing CreateTablet for tablet ff8e487fd51f4aa18bf59e5f8e011dcd (DEFAULT_TABLE table=default.Table_5592094c6a024bd6ac90af5c18b0daab [id=899506754d9140888a566b45450f2914]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.216432 28797 raft_consensus.cc:359] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.216634 28797 raft_consensus.cc:385] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.216743 28797 raft_consensus.cc:740] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.217236 28797 consensus_queue.cc:260] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.217396 28377 tablet_service.cc:1511] Processing CreateTablet for tablet 10e9a417eecf4d33ac3bd9bd6325f447 (DEFAULT_TABLE table=default.Table_819e5138e3c7415a80631f3cbb6f8b81 [id=0de5461053eb4517b15d6b91875fc04d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.217742 28797 ts_tablet_manager.cc:1434] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:31.217907 28387 tablet_service.cc:1511] Processing CreateTablet for tablet d8a71e334ed1400d97c965b49b677a81 (DEFAULT_TABLE table=default.Table_76b9ce534bf440919c76e1c8005ecc57 [id=445f1af32571418e979bbbbff739c798]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.218027 28797 tablet_bootstrap.cc:492] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:31.216787 28381 tablet_service.cc:1511] Processing CreateTablet for tablet 5115a31a38db48c1a6474917aff97c8e (DEFAULT_TABLE table=default.Table_94aa74339cf547a080b6ede6a36b4bed [id=9006108b57df43d8b559708d300e79ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.218508 28799 ts_tablet_manager.cc:1403] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.120s user 0.003s sys 0.002s
I20260618 13:59:31.219194 29644 consensus_queue.cc:1048] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:31.219223 28662 tablet_service.cc:1511] Processing CreateTablet for tablet ff8e487fd51f4aa18bf59e5f8e011dcd (DEFAULT_TABLE table=default.Table_5592094c6a024bd6ac90af5c18b0daab [id=899506754d9140888a566b45450f2914]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:31.220229 28483 leader_election.cc:343] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: e025b64380314546af8a1ab90c817825
I20260618 13:59:31.220378 28483 leader_election.cc:304] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:31.221027 28797 tablet_bootstrap.cc:654] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.221546 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0463fa2e30554abcbfc8f37b702e81d4" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:31.222543 29221 raft_consensus.cc:2749] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:31.222656 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0463fa2e30554abcbfc8f37b702e81d4" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:31.223009 28799 raft_consensus.cc:359] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.225251 28377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10e9a417eecf4d33ac3bd9bd6325f447. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.228420 28385 tablet_service.cc:1511] Processing CreateTablet for tablet a8da96fc621c47d4a444b5a39259f85d (DEFAULT_TABLE table=default.Table_a7ff69763fdb4ff3b26b180bf05907d9 [id=e39e2c5718794a748361d7b453c65404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.229010 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_12d6cc95071949c6a7a297cd08b5f898 [id=840c933184144aebb3659d939f679f00]). Expected version 1 got 0
I20260618 13:59:31.229473 29644 raft_consensus.cc:493] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.229635 29644 raft_consensus.cc:515] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.229701 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:31.230201 29644 leader_election.cc:290] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.230239 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:31.231755 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aef7c1f111634c98af046e73fa9ffdeb" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:31.232182 28512 tablet_service.cc:1511] Processing CreateTablet for tablet 60b6216474b74a14af6abf977e312b77 (DEFAULT_TABLE table=default.Table_72e5809d8cce4f3bbe8227efcb8e2b2c [id=dcd71daa60ee430d9ca46a1463d9a1bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.233103 28375 tablet_service.cc:1511] Processing CreateTablet for tablet a662bb37fb7d4df49e61e524808b67a8 (DEFAULT_TABLE table=default.Table_0a85d79af65f48a3803bfd1248593a11 [id=2f4d62be772a421ab777fd23be58b719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.233485 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46bfe8cc2078414d84e65e22ed4ef8a2" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:31.234077 28483 leader_election.cc:304] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:31.234217 28663 tablet_service.cc:1511] Processing CreateTablet for tablet 60b6216474b74a14af6abf977e312b77 (DEFAULT_TABLE table=default.Table_72e5809d8cce4f3bbe8227efcb8e2b2c [id=dcd71daa60ee430d9ca46a1463d9a1bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.234491 29221 raft_consensus.cc:2749] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:31.235080 28372 tablet_service.cc:1511] Processing CreateTablet for tablet fea86899d2c047d997cebe80551e83c9 (DEFAULT_TABLE table=default.Table_539ac216b6f4478eb994218d7d7980a4 [id=baa3214376034e9cae926def4c46b91e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.235292 28388 tablet_service.cc:1511] Processing CreateTablet for tablet 2e493f647a344cd581515a77230d765f (DEFAULT_TABLE table=default.Table_7699960fdee24242bf51bcfcbd6f68c2 [id=3b9ab803536344d3a59ff77b2b58ae31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.237563 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
W20260618 13:59:31.237893 28618 leader_election.cc:343] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.238658 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet cea4bc13eabc4dd98b4f96723498014c (table default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c]). Expected version 1 got 0
I20260618 13:59:31.238838 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 39b06ae7444a48dbbec9e096fd7e4052 (table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15]). Expected version 2 got 0
I20260618 13:59:31.239077 28515 tablet_service.cc:1511] Processing CreateTablet for tablet 8bef52c0fc354d6cb5388700641af9f8 (DEFAULT_TABLE table=default.Table_dc9c9ba6a1ce4446b3654e271445bd3a [id=adaa111ff5b74c8898cf8dabc33d621f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.239204 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet f278d0acbb0c4a0ca61640bc142e6335 (table default.Table_8783a8d8c526428281436b62b2581a33 [id=0d83fc68255c458587390c58b12e00d7]). Expected version 1 got 0
W20260618 13:59:31.239848 28346 leader_election.cc:343] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:31.240088 28346 leader_election.cc:343] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.241665 28659 tablet_service.cc:1511] Processing CreateTablet for tablet 8bef52c0fc354d6cb5388700641af9f8 (DEFAULT_TABLE table=default.Table_dc9c9ba6a1ce4446b3654e271445bd3a [id=adaa111ff5b74c8898cf8dabc33d621f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.242118 28346 leader_election.cc:304] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:31.242427 29522 raft_consensus.cc:2749] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:31.242514 28799 raft_consensus.cc:385] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.242611 28799 raft_consensus.cc:740] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.243813 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (DEFAULT_TABLE table=default.Table_b703123b497f445b89066c7ec657f91c [id=3bb8ec262af24e49896eb550e479c88f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.245311 28652 tablet_service.cc:1511] Processing CreateTablet for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (DEFAULT_TABLE table=default.Table_b703123b497f445b89066c7ec657f91c [id=3bb8ec262af24e49896eb550e479c88f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.246529 28510 tablet_service.cc:1511] Processing CreateTablet for tablet c25fa765ec794c34b0aa43761961a8c9 (DEFAULT_TABLE table=default.Table_2dc7e2d92c504d7f91ac02de7c2bdee4 [id=a7d18bd3a9474b92b0612b755f6124cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.240347 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aef7c1f111634c98af046e73fa9ffdeb" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:31.247567 28799 consensus_queue.cc:260] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.248298 28656 tablet_service.cc:1511] Processing CreateTablet for tablet c25fa765ec794c34b0aa43761961a8c9 (DEFAULT_TABLE table=default.Table_2dc7e2d92c504d7f91ac02de7c2bdee4 [id=a7d18bd3a9474b92b0612b755f6124cc]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:31.250046 28988 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet bf5e04eb43ed4cf493c0f6f08e22baeb consensus state has no leader
I20260618 13:59:31.256516 28654 tablet_service.cc:1511] Processing CreateTablet for tablet 2e493f647a344cd581515a77230d765f (DEFAULT_TABLE table=default.Table_7699960fdee24242bf51bcfcbd6f68c2 [id=3b9ab803536344d3a59ff77b2b58ae31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.257525 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bef52c0fc354d6cb5388700641af9f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.258029 28652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 038ea0e9c0b04db8b7ec56b065e3cc51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.258718 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e493f647a344cd581515a77230d765f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.261359 29402 raft_consensus.cc:515] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:31.262148 29402 leader_election.cc:290] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.262475 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fea86899d2c047d997cebe80551e83c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.262761 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a42aa779ae452e9420d730bf8d7d55" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:31.263000 28649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a662bb37fb7d4df49e61e524808b67a8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:31.263345 28618 leader_election.cc:343] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.263424 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8da96fc621c47d4a444b5a39259f85d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.263898 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff8e487fd51f4aa18bf59e5f8e011dcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.264372 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60b6216474b74a14af6abf977e312b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.266832 29522 raft_consensus.cc:493] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.267012 29522 raft_consensus.cc:515] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.267587 29522 leader_election.cc:290] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.268661 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c25fa765ec794c34b0aa43761961a8c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.273844 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:31.273728 28798 tablet_bootstrap.cc:654] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.278810 28381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5115a31a38db48c1a6474917aff97c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.280236 28384 tablet_service.cc:1511] Processing CreateTablet for tablet ff8e487fd51f4aa18bf59e5f8e011dcd (DEFAULT_TABLE table=default.Table_5592094c6a024bd6ac90af5c18b0daab [id=899506754d9140888a566b45450f2914]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.280443 29489 raft_consensus.cc:493] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.281978 28514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8da96fc621c47d4a444b5a39259f85d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.282470 28515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bef52c0fc354d6cb5388700641af9f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.283486 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a662bb37fb7d4df49e61e524808b67a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.283735 28510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c25fa765ec794c34b0aa43761961a8c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.284862 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 038ea0e9c0b04db8b7ec56b065e3cc51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.284931 29570 ts_tablet_manager.cc:1929] T 897bf6d6bbee4ad89c296325d94a3e53 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:31.285447 29570 log.cc:1199] T 897bf6d6bbee4ad89c296325d94a3e53 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/897bf6d6bbee4ad89c296325d94a3e53
I20260618 13:59:31.286293 28512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60b6216474b74a14af6abf977e312b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.288125 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e493f647a344cd581515a77230d765f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.289888 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 8bef52c0fc354d6cb5388700641af9f8 (DEFAULT_TABLE table=default.Table_dc9c9ba6a1ce4446b3654e271445bd3a [id=adaa111ff5b74c8898cf8dabc33d621f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.294225 28519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fea86899d2c047d997cebe80551e83c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.297204 29570 ts_tablet_manager.cc:1950] T 897bf6d6bbee4ad89c296325d94a3e53 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:31.299922 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff8e487fd51f4aa18bf59e5f8e011dcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.303102 28513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e493f647a344cd581515a77230d765f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.310696 29456 raft_consensus.cc:493] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.310871 29456 raft_consensus.cc:515] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.311678 29456 leader_election.cc:290] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:31.317945 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591465e2b57247a2b224cc8b09225811" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:31.318825 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bef52c0fc354d6cb5388700641af9f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.319154 28372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fea86899d2c047d997cebe80551e83c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.319345 28374 tablet_service.cc:1511] Processing CreateTablet for tablet 60b6216474b74a14af6abf977e312b77 (DEFAULT_TABLE table=default.Table_72e5809d8cce4f3bbe8227efcb8e2b2c [id=dcd71daa60ee430d9ca46a1463d9a1bf]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:31.322683 28483 leader_election.cc:343] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 125bb6885ab348d885d5a3b4bdd45bc7
I20260618 13:59:31.319875 29489 raft_consensus.cc:515] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.324416 28386 tablet_service.cc:1511] Processing CreateTablet for tablet c25fa765ec794c34b0aa43761961a8c9 (DEFAULT_TABLE table=default.Table_2dc7e2d92c504d7f91ac02de7c2bdee4 [id=a7d18bd3a9474b92b0612b755f6124cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.327279 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 897bf6d6bbee4ad89c296325d94a3e53 (table default.Table_35260f90f6814fa3915b4280cff96d1b [id=02ffed3d852845d6845fb5176f036fce]) successfully deleted
W20260618 13:59:31.331614 28346 leader_election.cc:343] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.333659 28375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a662bb37fb7d4df49e61e524808b67a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.334319 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92ddbdfdfff547588a03f453e413af8f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:31.331032 28391 tablet_service.cc:1511] Processing CreateTablet for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (DEFAULT_TABLE table=default.Table_b703123b497f445b89066c7ec657f91c [id=3bb8ec262af24e49896eb550e479c88f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:31.337807 28346 leader_election.cc:343] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.338023 28405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a42aa779ae452e9420d730bf8d7d55" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:31.338483 29489 leader_election.cc:290] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.339548 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff8e487fd51f4aa18bf59e5f8e011dcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.339823 28387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8a71e334ed1400d97c965b49b677a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.339905 28405 raft_consensus.cc:3060] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.342475 28374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60b6216474b74a14af6abf977e312b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.342715 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8da96fc621c47d4a444b5a39259f85d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.351502 28391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 038ea0e9c0b04db8b7ec56b065e3cc51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.352607 28596 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
W20260618 13:59:31.356220 28484 leader_election.cc:343] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 125bb6885ab348d885d5a3b4bdd45bc7
I20260618 13:59:31.356364 28484 leader_election.cc:304] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:31.357885 29456 raft_consensus.cc:2749] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:31.375373 29402 consensus_queue.cc:1048] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:31.379542 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591465e2b57247a2b224cc8b09225811" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:31.382853 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92ddbdfdfff547588a03f453e413af8f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:31.382040 29402 consensus_queue.cc:1048] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
W20260618 13:59:31.388168 28346 leader_election.cc:343] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.388306 28346 leader_election.cc:304] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:31.293797 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5115a31a38db48c1a6474917aff97c8e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:31.388787 28346 leader_election.cc:343] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.399062 28547 raft_consensus.cc:3060] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.404635 28515 tablet_service.cc:1511] Processing CreateTablet for tablet e196dd3c7fe14524a50152643ddbc2ad (DEFAULT_TABLE table=default.Table_35260f90f6814fa3915b4280cff96d1b [id=02ffed3d852845d6845fb5176f036fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.405213 28515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e196dd3c7fe14524a50152643ddbc2ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.412695 28346 leader_election.cc:304] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:31.413491 29489 raft_consensus.cc:2749] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:31.414472 28525 tablet_service.cc:1511] Processing CreateTablet for tablet ac9f580e35af4d4080d313b0283c0b7e (DEFAULT_TABLE table=default.Table_3fa48c06a84c475faa51492f5328d840 [id=1efa390c1c6041ca9d18c173e647f718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.415161 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c25fa765ec794c34b0aa43761961a8c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.415691 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac9f580e35af4d4080d313b0283c0b7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.417232 29489 raft_consensus.cc:2749] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:31.420710 28799 ts_tablet_manager.cc:1434] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.174s user 0.014s sys 0.002s
I20260618 13:59:31.421307 29679 raft_consensus.cc:493] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.422048 28373 tablet_service.cc:1511] Processing CreateTablet for tablet ac9f580e35af4d4080d313b0283c0b7e (DEFAULT_TABLE table=default.Table_3fa48c06a84c475faa51492f5328d840 [id=1efa390c1c6041ca9d18c173e647f718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.422676 28799 tablet_bootstrap.cc:492] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:31.424130 28798 tablet_bootstrap.cc:492] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:31.424377 28798 ts_tablet_manager.cc:1403] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.245s user 0.001s sys 0.011s
I20260618 13:59:31.425278 28798 raft_consensus.cc:359] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.425624 28798 raft_consensus.cc:385] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.425758 28798 raft_consensus.cc:740] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.421479 29679 raft_consensus.cc:515] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.429514 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b95bdb0e36584d92b9c28de257553827" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:31.429919 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b95bdb0e36584d92b9c28de257553827" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:31.431977 29456 raft_consensus.cc:493] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.432134 29456 raft_consensus.cc:515] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:31.432708 29456 leader_election.cc:290] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:31.434131 28484 leader_election.cc:343] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 68b8293a7c924a449307ab88a1cf9290
I20260618 13:59:31.434789 29221 raft_consensus.cc:493] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.434928 29221 raft_consensus.cc:515] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.435797 29221 leader_election.cc:290] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:31.436532 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc36fbea853a4d4e9f61909200239aed" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:31.437659 29303 consensus_queue.cc:1048] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:31.441607 28373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac9f580e35af4d4080d313b0283c0b7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.442672 28377 tablet_service.cc:1511] Processing CreateTablet for tablet e196dd3c7fe14524a50152643ddbc2ad (DEFAULT_TABLE table=default.Table_35260f90f6814fa3915b4280cff96d1b [id=02ffed3d852845d6845fb5176f036fce]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:31.446688 28346 leader_election.cc:343] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.448657 28649 tablet_service.cc:1511] Processing CreateTablet for tablet ac9f580e35af4d4080d313b0283c0b7e (DEFAULT_TABLE table=default.Table_3fa48c06a84c475faa51492f5328d840 [id=1efa390c1c6041ca9d18c173e647f718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.449074 28649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac9f580e35af4d4080d313b0283c0b7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.450865 28657 tablet_service.cc:1511] Processing CreateTablet for tablet e196dd3c7fe14524a50152643ddbc2ad (DEFAULT_TABLE table=default.Table_35260f90f6814fa3915b4280cff96d1b [id=02ffed3d852845d6845fb5176f036fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:31.451349 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e196dd3c7fe14524a50152643ddbc2ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:31.456717 28405 raft_consensus.cc:2468] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
W20260618 13:59:31.457080 28346 leader_election.cc:343] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.457211 28346 leader_election.cc:304] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:31.457609 29489 raft_consensus.cc:2749] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:31.457815 28377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e196dd3c7fe14524a50152643ddbc2ad. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:31.458148 28483 leader_election.cc:343] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 68b8293a7c924a449307ab88a1cf9290
I20260618 13:59:31.458307 28483 leader_election.cc:304] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:31.459030 29221 raft_consensus.cc:2749] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:31.462306 28618 consensus_peers.cc:597] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:31.426055 28798 consensus_queue.cc:260] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.468187 28798 ts_tablet_manager.cc:1434] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.044s user 0.001s sys 0.001s
I20260618 13:59:31.468554 28798 tablet_bootstrap.cc:492] T 7d4144b1b5ea4e968672616bc22bead8 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:31.485332 29303 raft_consensus.cc:493] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.485494 29303 raft_consensus.cc:515] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.486104 29303 leader_election.cc:290] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.491200 28798 tablet_bootstrap.cc:654] T 7d4144b1b5ea4e968672616bc22bead8 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.494210 28798 tablet_bootstrap.cc:492] T 7d4144b1b5ea4e968672616bc22bead8 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:31.496038 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc36fbea853a4d4e9f61909200239aed" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:31.496244 28404 raft_consensus.cc:2468] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:31.496752 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15180aefba0544329daee74308ef5010" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:31.496922 28410 raft_consensus.cc:2468] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:31.497543 28617 leader_election.cc:304] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:31.498399 29607 raft_consensus.cc:2804] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:31.498558 29607 raft_consensus.cc:697] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:31.499218 29607 consensus_queue.cc:237] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:31.501401 29489 raft_consensus.cc:493] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.501539 29489 raft_consensus.cc:515] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.502087 29489 leader_election.cc:290] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.504212 28483 leader_election.cc:304] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2; no voters: d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:31.504341 29489 raft_consensus.cc:493] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.504459 29489 raft_consensus.cc:515] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.504640 29456 raft_consensus.cc:2804] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260618 13:59:31.504736 29456 raft_consensus.cc:493] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:31.504868 29456 raft_consensus.cc:3060] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Advancing to term 2
I20260618 13:59:31.505276 28617 leader_election.cc:304] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
W20260618 13:59:31.505640 28617 consensus_peers.cc:597] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 233040a16b0d4b61b977ca57c1afec2d. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:31.505774 29640 consensus_queue.cc:1048] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:31.506201 29640 raft_consensus.cc:2804] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:31.506274 29640 raft_consensus.cc:493] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:31.506335 29640 raft_consensus.cc:3060] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:31.506546 29476 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 13:59:31.506961 29476 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 13:59:31.507062 29476 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260618 13:59:31.508265 28797 tablet_bootstrap.cc:492] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:31.508495 28797 ts_tablet_manager.cc:1403] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.291s user 0.010s sys 0.011s
I20260618 13:59:31.509438 28797 raft_consensus.cc:359] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:31.509680 28797 raft_consensus.cc:385] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.509784 28797 raft_consensus.cc:740] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.510108 28797 consensus_queue.cc:260] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:31.510797 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0039a463ec4a9c82a21a3a69b565fc" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:31.511132 28797 ts_tablet_manager.cc:1434] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:31.511454 28797 tablet_bootstrap.cc:492] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:31.511677 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0039a463ec4a9c82a21a3a69b565fc" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:31.494388 28798 ts_tablet_manager.cc:1403] T 7d4144b1b5ea4e968672616bc22bead8 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.002s
W20260618 13:59:31.513669 28346 leader_election.cc:343] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:31.513900 28346 leader_election.cc:343] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.513855 28798 raft_consensus.cc:359] T 7d4144b1b5ea4e968672616bc22bead8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.514042 28798 raft_consensus.cc:385] T 7d4144b1b5ea4e968672616bc22bead8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.514210 28798 raft_consensus.cc:740] T 7d4144b1b5ea4e968672616bc22bead8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.515666 28798 consensus_queue.cc:260] T 7d4144b1b5ea4e968672616bc22bead8 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.513976 28346 leader_election.cc:304] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:31.518834 28798 ts_tablet_manager.cc:1434] T 7d4144b1b5ea4e968672616bc22bead8 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.007s user 0.001s sys 0.002s
I20260618 13:59:31.519678 29679 leader_election.cc:290] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.521574 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4cfca9e76e641a7bc0b58b9ee240f36" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:31.522114 28797 tablet_bootstrap.cc:654] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.522670 29522 raft_consensus.cc:2749] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:31.522693 28458 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:31.523722 28799 tablet_bootstrap.cc:654] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.525218 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:31.525681 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_efb2b8e451084b7ba0de07e0a6e538c2 [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 1 got 0
I20260618 13:59:31.526046 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 9c17e76ae9b44207998a43e3576908c4 (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]). Expected version 2 got 0
W20260618 13:59:31.522348 28346 leader_election.cc:343] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.527490 29566 ts_tablet_manager.cc:1929] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:31.527602 29566 log.cc:1199] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/4a01dfb4d59549399c94640f16045555
I20260618 13:59:31.527473 29456 raft_consensus.cc:515] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.527954 29566 ts_tablet_manager.cc:1950] T 4a01dfb4d59549399c94640f16045555 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:31.528198 29640 raft_consensus.cc:515] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.528260 29456 leader_election.cc:290] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:31.528756 29640 leader_election.cc:290] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.532109 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 9) took 1164 ms. Trace:
I20260618 13:59:31.532222 29566 rpcz_store.cc:276] 0618 13:59:30.367787 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:30.367826 (+ 39us) service_pool.cc:224] Handling call
0618 13:59:31.091611 (+723785us) ts_tablet_manager.cc:1151] Deleting tablet 4a01dfb4d59549399c94640f16045555
0618 13:59:31.091622 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:31.257449 (+165827us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:31.527403 (+269954us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:31.532089 (+ 4686us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":723615}
I20260618 13:59:31.532588 29566 tablet_replica.cc:333] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:31.532766 29566 raft_consensus.cc:2243] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:31.533037 29566 raft_consensus.cc:2272] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:31.533509 29566 ts_tablet_manager.cc:1916] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:31.537451 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 4a01dfb4d59549399c94640f16045555 (table default.Table_7700ee6bae434a01b621ad02088cc5f0 [id=fe7eabed8da6400bbb721520c6182f8c]) successfully deleted
I20260618 13:59:31.537497 28798 tablet_bootstrap.cc:492] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:31.537870 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4cfca9e76e641a7bc0b58b9ee240f36" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:31.526369 29011 catalog_manager.cc:5696] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:31.539276 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 98b42d23832545ae81a7cdff19e4689f (table default.Table_c99584ecc88948a097b39c22d68b3938 [id=7c39b3c432e944bca05f7b2070a79ff9]). Expected version 1 got 0
I20260618 13:59:31.539642 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:31.540198 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:31.540994 29011 catalog_manager.cc:5696] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:31.541440 29489 leader_election.cc:290] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.541528 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 9b75ece59d924e3db49e002bf0faa89c (table default.Table_031e1a03030b45c99c1270d612008f91 [id=387d3ff1762749efad926181fe4c746a]). Expected version 1 got 0
I20260618 13:59:31.543376 29402 consensus_queue.cc:1048] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:31.545789 29476 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 13:59:31.545917 29476 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260618 13:59:31.546023 29476 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 13:59:31.546115 29476 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260618 13:59:31.547271 28798 tablet_bootstrap.cc:654] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:31.547889 28346 leader_election.cc:343] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.548082 28346 leader_election.cc:304] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:31.550299 28797 tablet_bootstrap.cc:492] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:31.550467 28797 ts_tablet_manager.cc:1403] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.039s user 0.004s sys 0.000s
I20260618 13:59:31.551834 28797 raft_consensus.cc:359] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.552114 28797 raft_consensus.cc:385] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.552179 28797 raft_consensus.cc:740] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.552430 28797 consensus_queue.cc:260] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.553364 28797 ts_tablet_manager.cc:1434] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:31.553681 28797 tablet_bootstrap.cc:492] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:31.554683 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15180aefba0544329daee74308ef5010" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:31.560359 28797 tablet_bootstrap.cc:654] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.561508 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15180aefba0544329daee74308ef5010" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:31.564221 29476 master_runner.cc:387] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.100.20:36797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.27.100.62:43001,127.27.100.61:45967,127.27.100.60:42083
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.100.60:42083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.100.60
--webserver_port=38293
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision faec34f53d4c9dd1e3a6dd2b7314d2fc7aadff6c
build type FASTDEBUG
built by None at 18 Jun 2026 13:43:31 UTC on bdcb31816ec0
build id 12454
ASAN enabled
I20260618 13:59:31.566612 29476 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 13:59:31.568982 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:31.569298 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 15180aefba0544329daee74308ef5010 (table default.Table_97b6bb39b3bf4612be391b9dd78d1646 [id=42a2a24590834abc9537669aaf223a7c]). Expected version 1 got 0
I20260618 13:59:31.569589 28403 raft_consensus.cc:3060] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.569942 29476 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20260618 13:59:31.569816 29011 catalog_manager.cc:5696] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:31.570276 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet dc36fbea853a4d4e9f61909200239aed (table default.Table_dbc6162c06f74e71b792bd1f7b2388fe [id=49576dabdecf41aca1b15ff3fc0c113c]). Expected version 6 got 0
I20260618 13:59:31.570536 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:31.570776 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:31.571199 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet a1a06078974e4bfbbc4b949802a39bf1 (table default.Table_f958c4ce58c34bfda98454664e202af2 [id=7b747ef3b2494ca78c499c5bb75648d4]). Expected version 2 got 0
I20260618 13:59:31.571399 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet ece7909d056c4ba8a5dca6423f0c1781 (table default.Table_26e7d8c4f91144268a2a43e83992667d [id=803d5220ce1a4dcfaf27d93c574be2ac]). Expected version 1 got 0
I20260618 13:59:31.571769 29011 catalog_manager.cc:5696] T e86aeb21427e4859a5266f4451469b25 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:31.572077 29489 raft_consensus.cc:2749] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:31.572480 29011 catalog_manager.cc:5696] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:31.572926 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 512a9e63ba324a30b8d8c37e585ebdf8 (table default.Table_c70bc1c98b3b49d7aa3080904667732d [id=901874cb80e74ba589679bb7b95c8fe4]). Expected version 1 got 0
I20260618 13:59:31.573352 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:31.573589 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc36fbea853a4d4e9f61909200239aed" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:31.573787 28402 raft_consensus.cc:3060] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Advancing to term 2
I20260618 13:59:31.573638 29011 catalog_manager.cc:5696] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:31.574029 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 6b8f273def3e4b7caa4d624cf87a106c (table default.Table_5159a242b12044fc9a0b3e535ea40050 [id=cfb4ca8c4e884ccc8602be52d0b39100]). Expected version 4 got 0
I20260618 13:59:31.574287 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 39b06ae7444a48dbbec9e096fd7e4052 (table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15]). Expected version 2 got 0
I20260618 13:59:31.561964 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc36fbea853a4d4e9f61909200239aed" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:31.576690 29402 raft_consensus.cc:493] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.576835 29402 raft_consensus.cc:515] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.573319 29607 consensus_queue.cc:1048] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:31.577513 29402 leader_election.cc:290] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.578821 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15180aefba0544329daee74308ef5010" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:31.580346 28547 raft_consensus.cc:3060] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.584637 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet f278d0acbb0c4a0ca61640bc142e6335 (table default.Table_8783a8d8c526428281436b62b2581a33 [id=0d83fc68255c458587390c58b12e00d7]). Expected version 1 got 0
I20260618 13:59:31.584652 28405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02b5cc08ca0c4e719e0414b345211ba5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:31.584959 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:31.585165 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet cea4bc13eabc4dd98b4f96723498014c (table default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c]). Expected version 1 got 0
I20260618 13:59:31.585368 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:31.585793 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:31.587730 28405 raft_consensus.cc:2468] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:31.588285 28617 leader_election.cc:304] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:31.588819 29607 raft_consensus.cc:2804] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:31.588981 29607 raft_consensus.cc:493] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:31.589152 29607 raft_consensus.cc:3060] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.592938 28797 tablet_bootstrap.cc:492] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:31.593137 28797 ts_tablet_manager.cc:1403] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.040s user 0.006s sys 0.000s
I20260618 13:59:31.593875 28798 tablet_bootstrap.cc:492] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:31.594089 28798 ts_tablet_manager.cc:1403] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.075s user 0.002s sys 0.005s
I20260618 13:59:31.594135 28797 raft_consensus.cc:359] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.594440 28797 raft_consensus.cc:385] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.594552 28797 raft_consensus.cc:740] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.594604 29402 consensus_queue.cc:1048] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:31.595120 28798 raft_consensus.cc:359] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.595249 28797 consensus_queue.cc:260] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.595395 28798 raft_consensus.cc:385] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.595484 28798 raft_consensus.cc:740] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.595753 28798 consensus_queue.cc:260] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.597481 28798 ts_tablet_manager.cc:1434] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260618 13:59:31.597785 28798 tablet_bootstrap.cc:492] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:31.598771 28797 ts_tablet_manager.cc:1434] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260618 13:59:31.599587 28797 tablet_bootstrap.cc:492] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:31.604913 28797 tablet_bootstrap.cc:654] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.605351 28798 tablet_bootstrap.cc:654] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.586097 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 1f81428f804a485c967026c4527fd250 (table default.Table_706eec6806a94d3f8d5b60708a0c1cda [id=b817db93d50144d7b7c238550eb53b10]). Expected version 1 got 0
I20260618 13:59:31.606913 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:31.607599 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:31.608537 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:31.609426 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:31.609815 29456 raft_consensus.cc:493] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.609972 29456 raft_consensus.cc:515] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.610605 29456 leader_election.cc:290] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:31.611359 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98b42d23832545ae81a7cdff19e4689f" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:31.616513 28799 tablet_bootstrap.cc:492] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:31.616864 28799 ts_tablet_manager.cc:1403] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.194s user 0.002s sys 0.002s
I20260618 13:59:31.617179 29640 raft_consensus.cc:493] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.617349 29640 raft_consensus.cc:515] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:31.619239 29413 raft_consensus.cc:493] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.619364 29413 raft_consensus.cc:515] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.619402 28799 raft_consensus.cc:359] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.619547 28799 raft_consensus.cc:385] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.619647 28799 raft_consensus.cc:740] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.620401 29413 leader_election.cc:290] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.620615 28798 tablet_bootstrap.cc:492] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:31.620803 28798 ts_tablet_manager.cc:1403] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.002s
I20260618 13:59:31.621042 28405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eea4d59935d40aa97dc10ec7469aa56" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:31.621300 28405 raft_consensus.cc:2468] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:31.620883 29640 leader_election.cc:290] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.620344 28799 consensus_queue.cc:260] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.621659 28617 leader_election.cc:304] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:31.621819 28798 raft_consensus.cc:359] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.622035 28798 raft_consensus.cc:385] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.622140 28798 raft_consensus.cc:740] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.622476 28799 ts_tablet_manager.cc:1434] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260618 13:59:31.622516 28798 consensus_queue.cc:260] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.623207 28798 ts_tablet_manager.cc:1434] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260618 13:59:31.623580 28798 tablet_bootstrap.cc:492] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:31.624756 29413 raft_consensus.cc:2804] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:31.624838 29413 raft_consensus.cc:493] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:31.624894 29413 raft_consensus.cc:3060] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.625412 28405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75824a5d2f7e47838b60e6a2b785c6b2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:31.625602 28405 raft_consensus.cc:2468] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:31.626488 28799 tablet_bootstrap.cc:492] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:31.627609 28548 raft_consensus.cc:1240] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [term 2 FOLLOWER]: Rejecting Update request from peer d6a169b16ced46ca9cfbc631709bb257 for earlier term 1. Current term is 2. Ops: []
I20260618 13:59:31.627592 28797 tablet_bootstrap.cc:492] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:31.622861 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:31.628185 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_12d6cc95071949c6a7a297cd08b5f898 [id=840c933184144aebb3659d939f679f00]). Expected version 1 got 0
I20260618 13:59:31.629060 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:31.629284 28797 ts_tablet_manager.cc:1403] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.000s
W20260618 13:59:31.629350 29704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 13:59:31.630800 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:31.633407 28798 tablet_bootstrap.cc:654] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:31.636382 29705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 13:59:31.644641 29607 raft_consensus.cc:515] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.645129 28798 tablet_bootstrap.cc:492] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:31.645407 28798 ts_tablet_manager.cc:1403] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.002s
I20260618 13:59:31.645510 29607 leader_election.cc:290] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:31.646188 29023 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7a9bd8577f4f4a9593a3ffeab582b02e consensus state has no leader
I20260618 13:59:31.646188 28402 raft_consensus.cc:2468] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 2.
I20260618 13:59:31.646435 29023 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 40) took 1317 ms. Trace:
I20260618 13:59:31.646600 28404 raft_consensus.cc:1240] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [term 2 FOLLOWER]: Rejecting Update request from peer d6a169b16ced46ca9cfbc631709bb257 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260618 13:59:31.646533 29023 rpcz_store.cc:276] 0618 13:59:30.328990 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:30.636748 (+307758us) service_pool.cc:224] Handling call
0618 13:59:30.637239 (+ 491us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:30.637451 (+ 212us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:30.637641 (+ 190us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:30.637735 (+ 94us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:30.881829 (+244094us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:31.646353 (+764524us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:30.773693 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:30.773808 (+ 115us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:30.773814 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:30.773816 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:30.773904 (+ 88us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:30.773914 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:30.773918 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:30.773972 (+ 54us) tablet.cc:701] Row locks acquired
0618 13:59:30.773975 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:30.774008 (+ 33us) write_op.cc:270] Start()
0618 13:59:30.774037 (+ 29us) write_op.cc:276] Timestamp: P: 1781791170773971 usec, L: 0
0618 13:59:30.774040 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:30.774232 (+ 192us) log.cc:844] Serialized 405 byte log entry
0618 13:59:30.879868 (+105636us) op_driver.cc:464] REPLICATION: finished
0618 13:59:30.880583 (+ 715us) write_op.cc:301] APPLY: starting
0618 13:59:30.880596 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:30.880626 (+ 30us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:30.880655 (+ 29us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:30.880691 (+ 36us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:30.880715 (+ 24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:30.880722 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:30.880773 (+ 51us) log.cc:844] Serialized 23 byte log entry
0618 13:59:30.880811 (+ 38us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:30.880824 (+ 13us) write_op.cc:454] Released schema lock
0618 13:59:30.880829 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:30.880886 (+ 57us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:30.880888 (+ 2us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":685,"apply.run_cpu_time_us":295,"apply.run_wall_time_us":295,"num_ops":1,"prepare.queue_time_us":135414,"prepare.run_cpu_time_us":685,"prepare.run_wall_time_us":3653,"raft.queue_time_us":849,"raft.run_cpu_time_us":8,"raft.run_wall_time_us":8,"replication_time_us":105810}]]}
I20260618 13:59:31.648125 28798 raft_consensus.cc:359] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.648317 28798 raft_consensus.cc:385] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.648383 28798 raft_consensus.cc:740] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.648619 28798 consensus_queue.cc:260] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.649401 28798 ts_tablet_manager.cc:1434] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 13:59:31.649684 28798 tablet_bootstrap.cc:492] T 512a9e63ba324a30b8d8c37e585ebdf8 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:31.650231 28483 leader_election.cc:304] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2; no voters:
I20260618 13:59:31.650552 29456 raft_consensus.cc:2804] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260618 13:59:31.650681 29456 raft_consensus.cc:697] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [term 2 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:31.650928 29456 consensus_queue.cc:237] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.653411 29476 server_base.cc:1061] running on GCE node
I20260618 13:59:31.655112 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02b5cc08ca0c4e719e0414b345211ba5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:31.655205 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75824a5d2f7e47838b60e6a2b785c6b2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:31.655898 28798 tablet_bootstrap.cc:654] T 512a9e63ba324a30b8d8c37e585ebdf8 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.656229 28617 leader_election.cc:304] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:31.656666 29303 raft_consensus.cc:2804] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:31.656757 29303 raft_consensus.cc:493] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:31.656843 29303 raft_consensus.cc:3060] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.661115 28797 raft_consensus.cc:359] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.661334 28797 raft_consensus.cc:385] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.661437 28797 raft_consensus.cc:740] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.661666 28797 consensus_queue.cc:260] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.662285 28799 tablet_bootstrap.cc:654] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.661986 29400 consensus_queue.cc:1059] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:31.662521 29400 raft_consensus.cc:3055] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Stepping down as leader of term 1
I20260618 13:59:31.662606 29400 raft_consensus.cc:740] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:31.662918 29400 consensus_queue.cc:260] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.663270 29400 raft_consensus.cc:3060] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Advancing to term 2
I20260618 13:59:31.664278 28797 ts_tablet_manager.cc:1434] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.035s user 0.002s sys 0.000s
I20260618 13:59:31.670266 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eea4d59935d40aa97dc10ec7469aa56" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:31.670753 28797 tablet_bootstrap.cc:492] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:31.674309 28797 tablet_bootstrap.cc:654] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.682108 28799 tablet_bootstrap.cc:492] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:31.684129 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02b5cc08ca0c4e719e0414b345211ba5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:31.684876 28799 ts_tablet_manager.cc:1403] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.058s user 0.001s sys 0.004s
I20260618 13:59:31.682963 28730 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
W20260618 13:59:31.690929 28618 leader_election.cc:343] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.699754 28799 raft_consensus.cc:359] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.701205 28799 raft_consensus.cc:385] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.701261 28406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02b5cc08ca0c4e719e0414b345211ba5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:31.701929 28799 raft_consensus.cc:740] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.701995 28406 raft_consensus.cc:3060] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.702159 28799 consensus_queue.cc:260] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.702764 28799 ts_tablet_manager.cc:1434] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260618 13:59:31.703071 28799 tablet_bootstrap.cc:492] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:31.706060 28797 tablet_bootstrap.cc:492] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:31.706212 28797 ts_tablet_manager.cc:1403] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.001s
I20260618 13:59:31.707217 28797 raft_consensus.cc:359] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:31.707422 28797 raft_consensus.cc:385] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.707492 28797 raft_consensus.cc:740] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.708689 28798 tablet_bootstrap.cc:492] T 512a9e63ba324a30b8d8c37e585ebdf8 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:31.708853 28798 ts_tablet_manager.cc:1403] T 512a9e63ba324a30b8d8c37e585ebdf8 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.059s user 0.008s sys 0.012s
I20260618 13:59:31.711529 29522 raft_consensus.cc:493] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.711686 29522 raft_consensus.cc:515] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.712407 29522 leader_election.cc:290] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.713367 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d523419ff244f085d6eac1a3231973" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:31.714242 28346 leader_election.cc:343] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.714887 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d523419ff244f085d6eac1a3231973" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:31.715256 28797 consensus_queue.cc:260] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:31.715781 28346 leader_election.cc:343] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.715904 28346 leader_election.cc:304] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:31.716379 29522 raft_consensus.cc:2749] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:31.717839 28799 tablet_bootstrap.cc:654] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.727330 28797 ts_tablet_manager.cc:1434] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.021s user 0.004s sys 0.000s
I20260618 13:59:31.728502 29456 raft_consensus.cc:493] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.728724 28797 tablet_bootstrap.cc:492] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:31.728696 29456 raft_consensus.cc:515] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.729352 29456 leader_election.cc:290] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:31.730185 29566 ts_tablet_manager.cc:1929] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:31.730278 29566 log.cc:1199] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/233040a16b0d4b61b977ca57c1afec2d
I20260618 13:59:31.730651 29566 ts_tablet_manager.cc:1950] T 233040a16b0d4b61b977ca57c1afec2d P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:31.730866 29413 raft_consensus.cc:515] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:31.731349 29413 leader_election.cc:290] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.731773 29413 raft_consensus.cc:493] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.731868 29413 raft_consensus.cc:515] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.732321 29413 leader_election.cc:290] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.735668 28798 raft_consensus.cc:359] T 512a9e63ba324a30b8d8c37e585ebdf8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.735849 28798 raft_consensus.cc:385] T 512a9e63ba324a30b8d8c37e585ebdf8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.735934 28798 raft_consensus.cc:740] T 512a9e63ba324a30b8d8c37e585ebdf8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.736289 28798 consensus_queue.cc:260] T 512a9e63ba324a30b8d8c37e585ebdf8 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.737372 28798 ts_tablet_manager.cc:1434] T 512a9e63ba324a30b8d8c37e585ebdf8 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.028s user 0.001s sys 0.001s
I20260618 13:59:31.737648 28798 tablet_bootstrap.cc:492] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:31.741067 28798 tablet_bootstrap.cc:654] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.732347 28799 tablet_bootstrap.cc:492] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:31.743465 28799 ts_tablet_manager.cc:1403] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.002s
I20260618 13:59:31.744202 28799 raft_consensus.cc:359] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.744351 28799 raft_consensus.cc:385] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.744393 28799 raft_consensus.cc:740] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.744565 28799 consensus_queue.cc:260] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.734550 28403 raft_consensus.cc:2468] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:31.745047 28799 ts_tablet_manager.cc:1434] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:31.745301 28799 tablet_bootstrap.cc:492] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:31.747128 28799 tablet_bootstrap.cc:654] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:31.751503 28484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.3:35503, user_credentials={real_user=slave}} blocked reactor thread for 51224us
I20260618 13:59:31.729926 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3dbf13544ce408ab97834aec5862390" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:31.751921 28799 tablet_bootstrap.cc:492] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:31.752036 28799 ts_tablet_manager.cc:1403] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260618 13:59:31.752635 28799 raft_consensus.cc:359] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.752794 28799 raft_consensus.cc:385] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.752851 28799 raft_consensus.cc:740] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.753494 28799 consensus_queue.cc:260] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.753957 28799 ts_tablet_manager.cc:1434] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:31.754189 28799 tablet_bootstrap.cc:492] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:31.754985 29710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 13:59:31.755688 28797 tablet_bootstrap.cc:654] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.756242 28799 tablet_bootstrap.cc:654] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.756510 29476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 13:59:31.759316 29476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 13:59:31.760540 29476 hybrid_clock.cc:648] HybridClock initialized: now 1781791171760377 us; error 145 us; skew 500 ppm
I20260618 13:59:31.761041 29476 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 13:59:31.763612 28797 tablet_bootstrap.cc:492] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:31.763767 28799 tablet_bootstrap.cc:492] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:31.763890 28799 ts_tablet_manager.cc:1403] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
I20260618 13:59:31.764494 28799 raft_consensus.cc:359] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.764653 28799 raft_consensus.cc:385] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.764698 28799 raft_consensus.cc:740] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.764891 28799 consensus_queue.cc:260] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.765290 28799 ts_tablet_manager.cc:1434] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260618 13:59:31.765865 28799 tablet_bootstrap.cc:492] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:31.766058 28797 ts_tablet_manager.cc:1403] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.002s
I20260618 13:59:31.766335 29476 webserver.cc:533] Webserver started at http://127.27.100.60:38293/ using document root <none> and password file <none>
I20260618 13:59:31.768247 29476 fs_manager.cc:362] Metadata directory not provided
I20260618 13:59:31.768344 28799 tablet_bootstrap.cc:654] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.768411 29476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 13:59:31.768743 28797 raft_consensus.cc:359] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.769357 28797 raft_consensus.cc:385] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.769558 28797 raft_consensus.cc:740] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.770109 28797 consensus_queue.cc:260] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.770951 29522 raft_consensus.cc:493] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.771071 29522 raft_consensus.cc:515] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.771634 29522 leader_election.cc:290] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.772193 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2df9e9e811ec4c2ea68467b1294bb62e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:31.772825 28346 leader_election.cc:343] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.780305 28798 tablet_bootstrap.cc:492] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:31.780613 28798 ts_tablet_manager.cc:1403] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.043s user 0.004s sys 0.000s
I20260618 13:59:31.781605 28798 raft_consensus.cc:359] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.781972 28798 raft_consensus.cc:385] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.782119 28798 raft_consensus.cc:740] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.783174 28799 tablet_bootstrap.cc:492] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:31.783303 28799 ts_tablet_manager.cc:1403] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.005s
I20260618 13:59:31.784029 28799 raft_consensus.cc:359] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.784194 28799 raft_consensus.cc:385] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.784245 28799 raft_consensus.cc:740] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.773394 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2df9e9e811ec4c2ea68467b1294bb62e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:31.784461 28799 consensus_queue.cc:260] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.785574 28799 ts_tablet_manager.cc:1434] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:31.785840 28799 tablet_bootstrap.cc:492] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:31.785780 28798 consensus_queue.cc:260] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.786304 28798 ts_tablet_manager.cc:1434] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260618 13:59:31.787096 28798 tablet_bootstrap.cc:492] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:31.787802 28799 tablet_bootstrap.cc:654] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.787909 28797 ts_tablet_manager.cc:1434] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.022s user 0.001s sys 0.001s
I20260618 13:59:31.788185 28797 tablet_bootstrap.cc:492] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:31.789196 28346 leader_election.cc:343] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.789335 28346 leader_election.cc:304] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:31.789583 29522 raft_consensus.cc:2749] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:31.796797 28797 tablet_bootstrap.cc:654] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.800052 28798 tablet_bootstrap.cc:654] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.802039 28799 tablet_bootstrap.cc:492] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:31.802176 28799 ts_tablet_manager.cc:1403] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.003s
I20260618 13:59:31.802887 28799 raft_consensus.cc:359] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:31.803059 28799 raft_consensus.cc:385] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.803107 28799 raft_consensus.cc:740] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.803303 28799 consensus_queue.cc:260] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:31.803748 28799 ts_tablet_manager.cc:1434] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:31.804008 28799 tablet_bootstrap.cc:492] T 3dd0eaa64271489d92aa380df877f257 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:31.805502 28799 tablet_bootstrap.cc:654] T 3dd0eaa64271489d92aa380df877f257 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:31.805645 28617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 103822us
I20260618 13:59:31.806802 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eea4d59935d40aa97dc10ec7469aa56" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:31.806983 28409 raft_consensus.cc:3060] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.808353 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:31.808373 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f278d0acbb0c4a0ca61640bc142e6335" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:31.808532 28403 raft_consensus.cc:2468] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:31.816350 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet dc36fbea853a4d4e9f61909200239aed (table default.Table_dbc6162c06f74e71b792bd1f7b2388fe [id=49576dabdecf41aca1b15ff3fc0c113c]). Expected version 6 got 0
I20260618 13:59:31.816584 29221 raft_consensus.cc:493] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.816748 29221 raft_consensus.cc:515] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.817399 29221 leader_election.cc:290] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:31.817665 29719 raft_consensus.cc:493] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.817823 29719 raft_consensus.cc:515] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.818694 29719 leader_election.cc:290] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.819064 29719 consensus_queue.cc:1048] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:31.819257 28799 tablet_bootstrap.cc:492] T 3dd0eaa64271489d92aa380df877f257 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:31.819406 28799 ts_tablet_manager.cc:1403] T 3dd0eaa64271489d92aa380df877f257 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.003s
I20260618 13:59:31.819723 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e95c88e4b0a046a3ad69ee49af47c7c3" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:31.820326 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
W20260618 13:59:31.820720 28617 consensus_peers.cc:597] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 897bf6d6bbee4ad89c296325d94a3e53. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:31.821275 28798 tablet_bootstrap.cc:492] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:31.821321 28799 raft_consensus.cc:359] T 3dd0eaa64271489d92aa380df877f257 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.821552 28798 ts_tablet_manager.cc:1403] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.006s
I20260618 13:59:31.821472 28799 raft_consensus.cc:385] T 3dd0eaa64271489d92aa380df877f257 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.821630 28799 raft_consensus.cc:740] T 3dd0eaa64271489d92aa380df877f257 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.821616 28617 leader_election.cc:304] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:31.821943 29719 raft_consensus.cc:2804] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:31.821871 28799 consensus_queue.cc:260] T 3dd0eaa64271489d92aa380df877f257 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.821945 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 15180aefba0544329daee74308ef5010 (table default.Table_97b6bb39b3bf4612be391b9dd78d1646 [id=42a2a24590834abc9537669aaf223a7c]). Expected version 1 got 0
I20260618 13:59:31.822053 29719 raft_consensus.cc:697] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:31.822129 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:31.822291 29719 consensus_queue.cc:237] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.822577 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 98b42d23832545ae81a7cdff19e4689f (table default.Table_c99584ecc88948a097b39c22d68b3938 [id=7c39b3c432e944bca05f7b2070a79ff9]). Expected version 1 got 0
I20260618 13:59:31.822788 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:31.822974 28799 ts_tablet_manager.cc:1434] T 3dd0eaa64271489d92aa380df877f257 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:31.823213 28799 tablet_bootstrap.cc:492] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:31.823314 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:31.823702 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:31.825023 28798 raft_consensus.cc:359] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.825593 28799 tablet_bootstrap.cc:654] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.826090 28798 raft_consensus.cc:385] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.826381 28798 raft_consensus.cc:740] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.826902 28798 consensus_queue.cc:260] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:31.827347 28618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 135735us
I20260618 13:59:31.829419 28798 ts_tablet_manager.cc:1434] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260618 13:59:31.832569 29303 raft_consensus.cc:515] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.833068 29303 leader_election.cc:290] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:31.833336 28406 raft_consensus.cc:2468] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:31.835125 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 10) took 1466 ms. Trace:
I20260618 13:59:31.835204 29566 rpcz_store.cc:276] 0618 13:59:30.368261 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:30.368299 (+ 38us) service_pool.cc:224] Handling call
0618 13:59:31.532544 (+1164245us) ts_tablet_manager.cc:1151] Deleting tablet 233040a16b0d4b61b977ca57c1afec2d
0618 13:59:31.532554 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:31.632338 (+ 99784us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:31.730120 (+ 97782us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:31.835110 (+104990us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1164067}
I20260618 13:59:31.835482 29566 tablet_replica.cc:333] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:31.835651 29566 raft_consensus.cc:2243] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:31.835777 29566 raft_consensus.cc:2272] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:31.836244 29566 ts_tablet_manager.cc:1916] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:31.837040 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 233040a16b0d4b61b977ca57c1afec2d (table default.Table_7e5df99a84c84d66b548875d6d934656 [id=2332167cba774a0795c11d97505049c9]) successfully deleted
I20260618 13:59:31.838524 28799 tablet_bootstrap.cc:492] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:31.838685 28799 ts_tablet_manager.cc:1403] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.003s
I20260618 13:59:31.839406 28799 raft_consensus.cc:359] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.839550 28799 raft_consensus.cc:385] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.839589 28799 raft_consensus.cc:740] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.839823 28799 consensus_queue.cc:260] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.840862 28799 ts_tablet_manager.cc:1434] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:31.841089 28799 tablet_bootstrap.cc:492] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:31.843740 28799 tablet_bootstrap.cc:654] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.829488 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eea4d59935d40aa97dc10ec7469aa56" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:31.830176 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f278d0acbb0c4a0ca61640bc142e6335" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:31.836699 29221 raft_consensus.cc:493] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.845994 29221 raft_consensus.cc:515] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.846767 29221 leader_election.cc:290] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:31.847020 28798 tablet_bootstrap.cc:492] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:31.838253 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba3d38d32c7c4721bad35f0da7a9ebe5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:31.847702 28483 leader_election.cc:343] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 0a51e830e3014726b1d1c2608973f036
I20260618 13:59:31.832181 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:31.853480 28799 tablet_bootstrap.cc:492] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:31.853615 28799 ts_tablet_manager.cc:1403] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260618 13:59:31.854012 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:31.854316 28799 raft_consensus.cc:359] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.854498 28799 raft_consensus.cc:385] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.854552 28799 raft_consensus.cc:740] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.854743 28799 consensus_queue.cc:260] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.855240 28799 ts_tablet_manager.cc:1434] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:31.855720 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet a1a06078974e4bfbbc4b949802a39bf1 (table default.Table_f958c4ce58c34bfda98454664e202af2 [id=7b747ef3b2494ca78c499c5bb75648d4]). Expected version 2 got 0
I20260618 13:59:31.855870 28799 tablet_bootstrap.cc:492] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:31.856272 28798 tablet_bootstrap.cc:654] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.857038 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98b42d23832545ae81a7cdff19e4689f" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:31.857468 28799 tablet_bootstrap.cc:654] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.857647 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet ece7909d056c4ba8a5dca6423f0c1781 (table default.Table_26e7d8c4f91144268a2a43e83992667d [id=803d5220ce1a4dcfaf27d93c574be2ac]). Expected version 1 got 0
I20260618 13:59:31.857759 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3dbf13544ce408ab97834aec5862390" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:31.858289 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9b75ece59d924e3db49e002bf0faa89c (table default.Table_031e1a03030b45c99c1270d612008f91 [id=387d3ff1762749efad926181fe4c746a]). Expected version 1 got 0
I20260618 13:59:31.858309 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e95c88e4b0a046a3ad69ee49af47c7c3" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:31.858728 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
W20260618 13:59:31.859457 28618 leader_election.cc:343] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:31.859697 28618 consensus_peers.cc:597] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:31.860333 28618 leader_election.cc:343] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:31.860670 28618 leader_election.cc:343] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:31.860958 28618 leader_election.cc:343] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:31.861250 28618 leader_election.cc:343] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:31.861826 28618 consensus_peers.cc:597] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:31.862843 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
W20260618 13:59:31.863595 28618 leader_election.cc:343] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:31.863750 28618 leader_election.cc:343] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:31.863917 28618 leader_election.cc:343] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:31.864329 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:31.867393 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 512a9e63ba324a30b8d8c37e585ebdf8 (table default.Table_c70bc1c98b3b49d7aa3080904667732d [id=901874cb80e74ba589679bb7b95c8fe4]). Expected version 1 got 0
I20260618 13:59:31.868307 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 6b8f273def3e4b7caa4d624cf87a106c (table default.Table_5159a242b12044fc9a0b3e535ea40050 [id=cfb4ca8c4e884ccc8602be52d0b39100]). Expected version 4 got 0
I20260618 13:59:31.869307 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet f278d0acbb0c4a0ca61640bc142e6335 (table default.Table_8783a8d8c526428281436b62b2581a33 [id=0d83fc68255c458587390c58b12e00d7]). Expected version 1 got 0
I20260618 13:59:31.870604 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 39b06ae7444a48dbbec9e096fd7e4052 (table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15]). Expected version 2 got 0
I20260618 13:59:31.871447 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:31.872246 28407 raft_consensus.cc:2468] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:31.872789 28617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 51189.9us
I20260618 13:59:31.873154 28617 leader_election.cc:304] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:31.873441 29303 raft_consensus.cc:2804] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:31.873503 29303 raft_consensus.cc:493] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:31.873553 29303 raft_consensus.cc:3060] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.874228 28617 leader_election.cc:304] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:31.874752 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:31.875469 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet cea4bc13eabc4dd98b4f96723498014c (table default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c]). Expected version 1 got 0
I20260618 13:59:31.876206 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:31.877328 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 1f81428f804a485c967026c4527fd250 (table default.Table_706eec6806a94d3f8d5b60708a0c1cda [id=b817db93d50144d7b7c238550eb53b10]). Expected version 1 got 0
I20260618 13:59:31.878322 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:31.879019 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:31.879472 29644 raft_consensus.cc:2804] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:31.879681 29644 raft_consensus.cc:697] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:31.879904 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:31.879972 29644 consensus_queue.cc:237] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.880661 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_efb2b8e451084b7ba0de07e0a6e538c2 [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 1 got 0
I20260618 13:59:31.882468 28617 leader_election.cc:304] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:31.881354 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:31.883277 29640 raft_consensus.cc:2804] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:31.883409 29640 raft_consensus.cc:493] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:31.883530 29640 raft_consensus.cc:3060] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.884074 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_12d6cc95071949c6a7a297cd08b5f898 [id=840c933184144aebb3659d939f679f00]). Expected version 1 got 0
I20260618 13:59:31.884974 28797 tablet_bootstrap.cc:492] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:31.885154 28797 ts_tablet_manager.cc:1403] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.097s user 0.009s sys 0.006s
I20260618 13:59:31.885949 28797 raft_consensus.cc:359] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.886184 28797 raft_consensus.cc:385] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.886281 28797 raft_consensus.cc:740] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.886519 28797 consensus_queue.cc:260] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.887565 28797 ts_tablet_manager.cc:1434] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:31.887981 28797 tablet_bootstrap.cc:492] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:31.889272 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba3d38d32c7c4721bad35f0da7a9ebe5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:31.893927 28407 raft_consensus.cc:2468] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:31.895782 28406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75824a5d2f7e47838b60e6a2b785c6b2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:31.895942 28406 raft_consensus.cc:3060] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.896799 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:31.898746 28617 leader_election.cc:304] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:31.899066 29719 raft_consensus.cc:2804] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:31.899147 29719 raft_consensus.cc:493] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:31.899199 29719 raft_consensus.cc:3060] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.900317 29018 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet c4cfca9e76e641a7bc0b58b9ee240f36
I20260618 13:59:31.900949 29018 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 35) took 1669 ms. Trace:
I20260618 13:59:31.901043 29018 rpcz_store.cc:276] 0618 13:59:30.231764 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:30.417721 (+185957us) service_pool.cc:224] Handling call
0618 13:59:30.418361 (+ 640us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:30.418572 (+ 211us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:30.418651 (+ 79us) catalog_manager.cc:2849] Locking tablets
0618 13:59:30.622372 (+203721us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:30.792830 (+170458us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:30.792927 (+ 97us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:31.900264 (+1107337us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:31.900891 (+ 627us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:30.746503 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:30.748560 (+ 2057us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:30.748567 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:30.748570 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:30.748696 (+ 126us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:30.749652 (+ 956us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:30.749661 (+ 9us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:30.749784 (+ 123us) tablet.cc:701] Row locks acquired
0618 13:59:30.749787 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:30.749840 (+ 53us) write_op.cc:270] Start()
0618 13:59:30.749894 (+ 54us) write_op.cc:276] Timestamp: P: 1781791170749800 usec, L: 0
0618 13:59:30.749900 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:30.751941 (+ 2041us) log.cc:844] Serialized 728 byte log entry
0618 13:59:30.786685 (+ 34744us) op_driver.cc:464] REPLICATION: finished
0618 13:59:30.787952 (+ 1267us) write_op.cc:301] APPLY: starting
0618 13:59:30.787970 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:30.787997 (+ 27us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:30.788027 (+ 30us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:30.788074 (+ 47us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:30.788099 (+ 25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:30.788104 (+ 5us) write_op.cc:312] APPLY: finished
0618 13:59:30.788151 (+ 47us) log.cc:844] Serialized 29 byte log entry
0618 13:59:30.788187 (+ 36us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:30.788199 (+ 12us) write_op.cc:454] Released schema lock
0618 13:59:30.788203 (+ 4us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:30.788263 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:30.788269 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1238,"apply.run_cpu_time_us":301,"apply.run_wall_time_us":301,"num_ops":2,"prepare.queue_time_us":123488,"prepare.run_cpu_time_us":1109,"prepare.run_wall_time_us":19923,"raft.queue_time_us":9286,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":12,"replication_time_us":36760}]]}
I20260618 13:59:31.901849 29018 master_service.cc:960] ReplaceTablet: received request to replace tablet c7f42997416841a7b2c1ad347d2e9282 from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:31.902683 28373 tablet_service.cc:1558] Processing DeleteTablet for tablet c4cfca9e76e641a7bc0b58b9ee240f36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:31.903075 28525 tablet_service.cc:1558] Processing DeleteTablet for tablet c4cfca9e76e641a7bc0b58b9ee240f36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:31.903246 29745 tablet_replica.cc:333] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:31.903332 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75824a5d2f7e47838b60e6a2b785c6b2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:31.903517 29745 raft_consensus.cc:2243] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:31.903661 29745 raft_consensus.cc:2272] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:31.904156 29745 ts_tablet_manager.cc:1916] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:31.904623 28548 raft_consensus.cc:3060] T 7d4144b1b5ea4e968672616bc22bead8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.911410 28798 tablet_bootstrap.cc:492] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:31.913103 28797 tablet_bootstrap.cc:654] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:31.913385 29017 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c38a2156a55e4e5eb91d101e9a61eecd consensus state has no leader
I20260618 13:59:31.913522 29019 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 3dd0eaa64271489d92aa380df877f257
I20260618 13:59:31.913585 29017 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 41) took 1584 ms. Trace:
I20260618 13:59:31.914116 29019 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 24) took 2181 ms. Trace:
I20260618 13:59:31.913703 29017 rpcz_store.cc:276] 0618 13:59:30.329117 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:30.881509 (+552392us) service_pool.cc:224] Handling call
0618 13:59:30.885044 (+ 3535us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:30.885280 (+ 236us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:30.885481 (+ 201us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:30.885568 (+ 87us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:30.897972 (+ 12404us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:31.913540 (+1015568us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:30.886128 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:30.886251 (+ 123us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:30.886257 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:30.886259 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:30.886366 (+ 107us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:30.886379 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:30.886383 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:30.886467 (+ 84us) tablet.cc:701] Row locks acquired
0618 13:59:30.886469 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:30.886506 (+ 37us) write_op.cc:270] Start()
0618 13:59:30.886540 (+ 34us) write_op.cc:276] Timestamp: P: 1781791170886455 usec, L: 0
0618 13:59:30.886544 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:30.886727 (+ 183us) log.cc:844] Serialized 405 byte log entry
0618 13:59:30.895618 (+ 8891us) op_driver.cc:464] REPLICATION: finished
0618 13:59:30.895797 (+ 179us) write_op.cc:301] APPLY: starting
0618 13:59:30.895812 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:30.895860 (+ 48us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:30.895891 (+ 31us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:30.895939 (+ 48us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:30.895979 (+ 40us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:30.895988 (+ 9us) write_op.cc:312] APPLY: finished
0618 13:59:30.896061 (+ 73us) log.cc:844] Serialized 23 byte log entry
0618 13:59:30.896117 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:30.896136 (+ 19us) write_op.cc:454] Released schema lock
0618 13:59:30.896143 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:30.897699 (+ 1556us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:30.897705 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":105,"apply.run_cpu_time_us":460,"apply.run_wall_time_us":1887,"num_ops":1,"prepare.queue_time_us":51,"prepare.run_cpu_time_us":687,"prepare.run_wall_time_us":687,"raft.queue_time_us":761,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":8,"replication_time_us":9040}]]}
I20260618 13:59:31.914235 29019 rpcz_store.cc:276] 0618 13:59:29.732177 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:30.118134 (+385957us) service_pool.cc:224] Handling call
0618 13:59:30.121988 (+ 3854us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb46856b6 00007fcbab0e2486 00007fcbab0e21b5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:30.122520 (+ 532us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:30.122684 (+ 164us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:30.122763 (+ 79us) catalog_manager.cc:2849] Locking tablets
0618 13:59:30.615238 (+492475us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:30.788709 (+173471us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:30.788800 (+ 91us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:31.913460 (+1124660us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:31.914074 (+ 614us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:30.717553 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:30.717679 (+ 126us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:30.717686 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:30.717688 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:30.717823 (+ 135us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:30.717837 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:30.717842 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:30.717958 (+ 116us) tablet.cc:701] Row locks acquired
0618 13:59:30.717960 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:30.718001 (+ 41us) write_op.cc:270] Start()
0618 13:59:30.718038 (+ 37us) write_op.cc:276] Timestamp: P: 1781791170717965 usec, L: 0
0618 13:59:30.718042 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:30.718270 (+ 228us) log.cc:844] Serialized 728 byte log entry
0618 13:59:30.786555 (+ 68285us) op_driver.cc:464] REPLICATION: finished
0618 13:59:30.786819 (+ 264us) write_op.cc:301] APPLY: starting
0618 13:59:30.786840 (+ 21us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:30.786908 (+ 68us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:30.786938 (+ 30us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:30.787024 (+ 86us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:30.787061 (+ 37us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:30.787068 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:30.787157 (+ 89us) log.cc:844] Serialized 29 byte log entry
0618 13:59:30.787224 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:30.787246 (+ 22us) write_op.cc:454] Released schema lock
0618 13:59:30.787253 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:30.787333 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:30.787337 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":8150272,"child_traces":[["op",{"apply.queue_time_us":211,"apply.run_cpu_time_us":504,"apply.run_wall_time_us":503,"num_ops":2,"prepare.queue_time_us":101766,"prepare.run_cpu_time_us":906,"prepare.run_wall_time_us":20686,"raft.queue_time_us":76,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":7,"replication_time_us":68472}]]}
I20260618 13:59:31.914947 29019 master_service.cc:960] ReplaceTablet: received request to replace tablet aa8cf927bed9437086efd8e0f21ba26a from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:31.916275 29020 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet e5f7eaa7d218440ca2c35340c5c87814
I20260618 13:59:31.916853 29020 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 27) took 2184 ms. Trace:
I20260618 13:59:31.916947 29020 rpcz_store.cc:276] 0618 13:59:29.732460 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:30.156183 (+423723us) service_pool.cc:224] Handling call
0618 13:59:30.239147 (+ 82964us) spinlock_profiling.cc:243] Waited 82.7 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb46856b6 00007fcbab0e2486 00007fcbab0e21b5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:30.239620 (+ 473us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:30.239763 (+ 143us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:30.239814 (+ 51us) catalog_manager.cc:2849] Locking tablets
0618 13:59:30.241154 (+ 1340us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:30.416463 (+175309us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:30.416562 (+ 99us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:31.916206 (+1499644us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:31.916785 (+ 579us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:30.351617 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:30.351715 (+ 98us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:30.351719 (+ 4us) write_op.cc:435] Acquired schema lock
0618 13:59:30.351721 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:30.351816 (+ 95us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:30.351824 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:30.351829 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:30.351926 (+ 97us) tablet.cc:701] Row locks acquired
0618 13:59:30.351928 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:30.351953 (+ 25us) write_op.cc:270] Start()
0618 13:59:30.351974 (+ 21us) write_op.cc:276] Timestamp: P: 1781791170351916 usec, L: 0
0618 13:59:30.351977 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:30.389506 (+ 37529us) log.cc:844] Serialized 728 byte log entry
0618 13:59:30.397487 (+ 7981us) spinlock_profiling.cc:243] Waited 37.3 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fcbac75ef22 00007fcbadb40305 00007fcbadb3ee29 00007fcbadb47250 00007fcbadb4705d 0000000000aa681c 00007fcba07299ff 00007fcba072cd1b 00007fcba072cb98 0000000000aa681c
0618 13:59:30.406592 (+ 9105us) spinlock_profiling.cc:243] Waited 3.17 ms on lock 0x616000032eb8. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fcbac693f2f 00007fcbac661c3b 00007fcbac66cbd1 00007fcbac66c8bd 0000000000aa681c 00007fcba07299ff 00007fcba072cd1b 00007fcba072cb98 0000000000aa681c 00007fcba0708f48
0618 13:59:30.414349 (+ 7757us) op_driver.cc:464] REPLICATION: finished
0618 13:59:30.414657 (+ 308us) write_op.cc:301] APPLY: starting
0618 13:59:30.414681 (+ 24us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:30.414746 (+ 65us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:30.414750 (+ 4us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:30.414843 (+ 93us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:30.414891 (+ 48us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:30.414903 (+ 12us) write_op.cc:312] APPLY: finished
0618 13:59:30.415011 (+ 108us) log.cc:844] Serialized 29 byte log entry
0618 13:59:30.415084 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:30.415106 (+ 22us) write_op.cc:454] Released schema lock
0618 13:59:30.415115 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:30.415206 (+ 91us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:30.415209 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":190234880,"child_traces":[["op",{"apply.queue_time_us":248,"apply.run_cpu_time_us":537,"apply.run_wall_time_us":537,"num_ops":2,"prepare.queue_time_us":107507,"prepare.run_cpu_time_us":1233,"prepare.run_wall_time_us":45942,"raft.queue_time_us":5416,"raft.run_cpu_time_us":827,"raft.run_wall_time_us":7622,"replication_time_us":62340,"spinlock_wait_cycles":93122304}]]}
I20260618 13:59:31.917796 29020 master_service.cc:960] ReplaceTablet: received request to replace tablet ba0d05ea44e1473fb4b24f61eab096e4 from {username='slave'} at 127.0.0.1:33144
W20260618 13:59:31.920410 29026 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b509f940a02545019c526e45e8372c97 consensus state has no leader
W20260618 13:59:31.920679 29007 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2df9e9e811ec4c2ea68467b1294bb62e consensus state has no leader
I20260618 13:59:31.918071 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 3dd0eaa64271489d92aa380df877f257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:31.918231 28512 tablet_service.cc:1558] Processing DeleteTablet for tablet e5f7eaa7d218440ca2c35340c5c87814 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:31.922883 28798 ts_tablet_manager.cc:1403] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.076s user 0.006s sys 0.002s
I20260618 13:59:31.924115 28798 raft_consensus.cc:359] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:31.926774 29019 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 44) took 1545 ms. Trace:
I20260618 13:59:31.926894 28373 tablet_service.cc:1558] Processing DeleteTablet for tablet 3dd0eaa64271489d92aa380df877f257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:31.926903 29019 rpcz_store.cc:276] 0618 13:59:30.381015 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:31.914871 (+1533856us) service_pool.cc:224] Handling call
0618 13:59:31.926675 (+ 11804us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260618 13:59:31.927386 29017 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_abe180e115a04052ae89cc92d3c1c1e6" } new_table_name: "default.Table_7f787ab851be496dae97cba8835db19b" modify_external_catalogs: true
I20260618 13:59:31.927434 29019 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_73f07f55ca3145b89d0559c0ff579856"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:31.929430 29019 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_73f07f55ca3145b89d0559c0ff579856 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:31.929881 29026 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 31) took 2197 ms. Trace:
I20260618 13:59:31.930001 29026 rpcz_store.cc:276] 0618 13:59:29.732820 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:30.294297 (+561477us) service_pool.cc:224] Handling call
0618 13:59:30.310792 (+ 16495us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:30.311028 (+ 236us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:30.311219 (+ 191us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:30.311284 (+ 65us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:30.454536 (+143252us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:31.929820 (+1475284us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:30.416985 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:30.417081 (+ 96us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:30.417085 (+ 4us) write_op.cc:435] Acquired schema lock
0618 13:59:30.417088 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:30.417174 (+ 86us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:30.417184 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:30.417188 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:30.417238 (+ 50us) tablet.cc:701] Row locks acquired
0618 13:59:30.417240 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:30.417269 (+ 29us) write_op.cc:270] Start()
0618 13:59:30.417293 (+ 24us) write_op.cc:276] Timestamp: P: 1781791170417224 usec, L: 0
0618 13:59:30.417296 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:30.417446 (+ 150us) log.cc:844] Serialized 405 byte log entry
0618 13:59:30.449495 (+ 32049us) op_driver.cc:464] REPLICATION: finished
0618 13:59:30.450341 (+ 846us) write_op.cc:301] APPLY: starting
0618 13:59:30.450353 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:30.450382 (+ 29us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:30.450405 (+ 23us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:30.450472 (+ 67us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:30.450497 (+ 25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:30.450504 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:30.450569 (+ 65us) log.cc:844] Serialized 23 byte log entry
0618 13:59:30.450616 (+ 47us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:30.450629 (+ 13us) write_op.cc:454] Released schema lock
0618 13:59:30.450634 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:30.450705 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:30.450709 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":807,"apply.run_cpu_time_us":356,"apply.run_wall_time_us":357,"num_ops":1,"prepare.queue_time_us":105208,"prepare.run_cpu_time_us":559,"prepare.run_wall_time_us":896,"raft.queue_time_us":40,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"replication_time_us":32177}]]}
I20260618 13:59:31.930909 29026 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_5f54daf83a3347cc885616848fee0f43"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:31.931396 29026 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f54daf83a3347cc885616848fee0f43 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:31.934175 28798 raft_consensus.cc:385] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:31.934266 28798 raft_consensus.cc:740] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:31.934556 28798 consensus_queue.cc:260] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:31.937121 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet c4cfca9e76e641a7bc0b58b9ee240f36 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c4cfca9e76e641a7bc0b58b9ee240f36 already in progress: creating tablet
W20260618 13:59:31.938388 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 3dd0eaa64271489d92aa380df877f257 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3dd0eaa64271489d92aa380df877f257 already in progress: creating tablet
I20260618 13:59:31.938519 29007 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 38) took 1672 ms. Trace:
W20260618 13:59:31.939018 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet e5f7eaa7d218440ca2c35340c5c87814 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5f7eaa7d218440ca2c35340c5c87814 already in progress: creating tablet
I20260618 13:59:31.938607 29007 rpcz_store.cc:276] 0618 13:59:30.266172 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:30.473948 (+207776us) service_pool.cc:224] Handling call
0618 13:59:30.569965 (+ 96017us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:30.570247 (+ 282us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:30.570550 (+ 303us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:30.570658 (+ 108us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:30.635862 (+ 65204us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:31.938456 (+1302594us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:30.584979 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:30.585115 (+ 136us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:30.585121 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:30.585123 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:30.585229 (+ 106us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:30.585243 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:30.585250 (+ 7us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:30.585323 (+ 73us) tablet.cc:701] Row locks acquired
0618 13:59:30.585327 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:30.585366 (+ 39us) write_op.cc:270] Start()
0618 13:59:30.585404 (+ 38us) write_op.cc:276] Timestamp: P: 1781791170585299 usec, L: 0
0618 13:59:30.585410 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:30.585609 (+ 199us) log.cc:844] Serialized 405 byte log entry
0618 13:59:30.611106 (+ 25497us) op_driver.cc:464] REPLICATION: finished
0618 13:59:30.612863 (+ 1757us) write_op.cc:301] APPLY: starting
0618 13:59:30.612874 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:30.612900 (+ 26us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:30.612920 (+ 20us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:30.612950 (+ 30us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:30.612973 (+ 23us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:30.612980 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:30.613027 (+ 47us) log.cc:844] Serialized 23 byte log entry
0618 13:59:30.613059 (+ 32us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:30.613072 (+ 13us) write_op.cc:454] Released schema lock
0618 13:59:30.613076 (+ 4us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:30.613131 (+ 55us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:30.613134 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1712,"apply.run_cpu_time_us":263,"apply.run_wall_time_us":263,"num_ops":1,"prepare.queue_time_us":13745,"prepare.run_cpu_time_us":744,"prepare.run_wall_time_us":1148,"raft.queue_time_us":40,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":25677}]]}
I20260618 13:59:31.939498 29018 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 42) took 1605 ms. Trace:
I20260618 13:59:31.939555 29007 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_5d485acde41448d984aa79fa58b1866b" } modify_external_catalogs: true
I20260618 13:59:31.939620 29018 rpcz_store.cc:276] 0618 13:59:30.333797 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:31.901757 (+1567960us) service_pool.cc:224] Handling call
0618 13:59:31.939456 (+ 37699us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260618 13:59:31.939723 29007 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_5d485acde41448d984aa79fa58b1866b" } modify_external_catalogs: true
I20260618 13:59:31.940014 29018 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_3a7cb505ade04e6fa187bf8690075057"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:31.940608 29018 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a7cb505ade04e6fa187bf8690075057 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:31.942783 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet e5f7eaa7d218440ca2c35340c5c87814 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:31.945771 28798 ts_tablet_manager.cc:1434] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.022s user 0.002s sys 0.000s
I20260618 13:59:31.946672 29221 raft_consensus.cc:493] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:31.946851 29221 raft_consensus.cc:515] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:31.947295 28798 tablet_bootstrap.cc:492] T 118259402d7d48dda381cdd18eec0f1b P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:31.947533 29221 leader_election.cc:290] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:31.948144 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fae928833e44d2ba860d6184d4c0142" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:31.952334 28409 raft_consensus.cc:2468] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:31.963143 28549 raft_consensus.cc:3060] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:31.966324 28484 leader_election.cc:304] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:31.970606 29019 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 46) took 1489 ms. Trace:
I20260618 13:59:31.970733 29019 rpcz_store.cc:276] 0618 13:59:30.480873 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:31.927186 (+1446313us) service_pool.cc:224] Handling call
0618 13:59:31.929810 (+ 2624us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:31.930353 (+ 543us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:31.970257 (+ 39904us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:31.970391 (+ 134us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:31.970532 (+ 141us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:31.930957 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:31.931085 (+ 128us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:31.931091 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:31.931093 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:31.931232 (+ 139us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:31.931246 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:31.931250 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:31.931335 (+ 85us) tablet.cc:701] Row locks acquired
0618 13:59:31.931337 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:31.931377 (+ 40us) write_op.cc:270] Start()
0618 13:59:31.931423 (+ 46us) write_op.cc:276] Timestamp: P: 1781791171931327 usec, L: 0
0618 13:59:31.931426 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:31.931641 (+ 215us) log.cc:844] Serialized 433 byte log entry
0618 13:59:31.969476 (+ 37835us) op_driver.cc:464] REPLICATION: finished
0618 13:59:31.969622 (+ 146us) write_op.cc:301] APPLY: starting
0618 13:59:31.969638 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:31.969677 (+ 39us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:31.969713 (+ 36us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:31.969853 (+ 140us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:31.969891 (+ 38us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:31.969903 (+ 12us) write_op.cc:312] APPLY: finished
0618 13:59:31.969984 (+ 81us) log.cc:844] Serialized 29 byte log entry
0618 13:59:31.970049 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:31.970066 (+ 17us) write_op.cc:454] Released schema lock
0618 13:59:31.970072 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:31.970140 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:31.970146 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":65408,"child_traces":[["op",{"apply.queue_time_us":97,"apply.run_cpu_time_us":507,"apply.run_wall_time_us":506,"num_ops":2,"prepare.queue_time_us":73,"prepare.run_cpu_time_us":799,"prepare.run_wall_time_us":799,"raft.queue_time_us":2579,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":566,"replication_time_us":38012,"spinlock_wait_cycles":694016}]]}
I20260618 13:59:31.972487 29018 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 49) took 1379 ms. Trace:
I20260618 13:59:31.972607 29018 rpcz_store.cc:276] 0618 13:59:30.593262 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:31.939804 (+1346542us) service_pool.cc:224] Handling call
0618 13:59:31.940704 (+ 900us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:31.941230 (+ 526us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:31.972261 (+ 31031us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:31.972371 (+ 110us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:31.972452 (+ 81us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:31.941865 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:31.941973 (+ 108us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:31.941979 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:31.941981 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:31.942068 (+ 87us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:31.942081 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:31.942089 (+ 8us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:31.942171 (+ 82us) tablet.cc:701] Row locks acquired
0618 13:59:31.942175 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:31.942214 (+ 39us) write_op.cc:270] Start()
0618 13:59:31.942246 (+ 32us) write_op.cc:276] Timestamp: P: 1781791171942164 usec, L: 0
0618 13:59:31.942249 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:31.942485 (+ 236us) log.cc:844] Serialized 433 byte log entry
0618 13:59:31.970928 (+ 28443us) op_driver.cc:464] REPLICATION: finished
0618 13:59:31.971342 (+ 414us) write_op.cc:301] APPLY: starting
0618 13:59:31.971355 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:31.971391 (+ 36us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:31.971435 (+ 44us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:31.971522 (+ 87us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:31.971546 (+ 24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:31.971552 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:31.971605 (+ 53us) log.cc:844] Serialized 29 byte log entry
0618 13:59:31.971651 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:31.971666 (+ 15us) write_op.cc:454] Released schema lock
0618 13:59:31.971671 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:31.971739 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:31.971746 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":361,"apply.run_cpu_time_us":392,"apply.run_wall_time_us":392,"num_ops":2,"prepare.queue_time_us":48,"prepare.run_cpu_time_us":714,"prepare.run_wall_time_us":715,"raft.queue_time_us":413,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":9,"replication_time_us":28652}]]}
I20260618 13:59:31.979168 29026 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 47) took 1483 ms. Trace:
I20260618 13:59:31.979537 28798 tablet_bootstrap.cc:654] T 118259402d7d48dda381cdd18eec0f1b P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:31.979274 29026 rpcz_store.cc:276] 0618 13:59:30.495691 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:31.930703 (+1435012us) service_pool.cc:224] Handling call
0618 13:59:31.931498 (+ 795us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:31.932013 (+ 515us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:31.978916 (+ 46903us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:31.979018 (+ 102us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:31.979104 (+ 86us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:31.951129 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:31.951231 (+ 102us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:31.951237 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:31.951240 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:31.951342 (+ 102us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:31.951353 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:31.951357 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:31.951428 (+ 71us) tablet.cc:701] Row locks acquired
0618 13:59:31.951430 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:31.951465 (+ 35us) write_op.cc:270] Start()
0618 13:59:31.951496 (+ 31us) write_op.cc:276] Timestamp: P: 1781791171951416 usec, L: 0
0618 13:59:31.951499 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:31.951675 (+ 176us) log.cc:844] Serialized 433 byte log entry
0618 13:59:31.970988 (+ 19313us) op_driver.cc:464] REPLICATION: finished
0618 13:59:31.971823 (+ 835us) write_op.cc:301] APPLY: starting
0618 13:59:31.971835 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:31.971872 (+ 37us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:31.971908 (+ 36us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:31.972013 (+ 105us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:31.972035 (+ 22us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:31.972041 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:31.972091 (+ 50us) log.cc:844] Serialized 29 byte log entry
0618 13:59:31.972136 (+ 45us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:31.972151 (+ 15us) write_op.cc:454] Released schema lock
0618 13:59:31.972156 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":789,"apply.run_cpu_time_us":395,"apply.run_wall_time_us":395,"num_ops":2,"prepare.queue_time_us":56,"prepare.run_cpu_time_us":653,"prepare.run_wall_time_us":653,"raft.queue_time_us":3941,"raft.run_cpu_time_us":8,"raft.run_wall_time_us":9,"replication_time_us":19474,"threads_started":1}]]}
I20260618 13:59:31.981207 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet c4cfca9e76e641a7bc0b58b9ee240f36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:31.981351 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet e5f7eaa7d218440ca2c35340c5c87814 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:31.981594 28617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 81477.1us
I20260618 13:59:31.981643 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 3dd0eaa64271489d92aa380df877f257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:31.982738 28617 leader_election.cc:304] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:31.983069 29721 raft_consensus.cc:2804] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:31.983197 29721 raft_consensus.cc:697] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:31.983426 29721 consensus_queue.cc:237] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:32.001852 29020 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 45) took 1571 ms. Trace:
I20260618 13:59:32.002027 29020 rpcz_store.cc:276] 0618 13:59:30.430268 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:31.917712 (+1487444us) service_pool.cc:224] Handling call
0618 13:59:32.001794 (+ 84082us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260618 13:59:32.003892 29714 consensus_queue.cc:1048] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:32.004392 28799 tablet_bootstrap.cc:492] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:32.004532 28799 ts_tablet_manager.cc:1403] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.149s user 0.058s sys 0.079s
I20260618 13:59:32.005384 28799 raft_consensus.cc:359] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.005575 28799 raft_consensus.cc:385] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.005654 28799 raft_consensus.cc:740] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.005892 28799 consensus_queue.cc:260] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.006459 28799 ts_tablet_manager.cc:1434] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260618 13:59:32.006723 28799 tablet_bootstrap.cc:492] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:32.013984 29522 raft_consensus.cc:493] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.014133 29522 raft_consensus.cc:515] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.015014 29522 leader_election.cc:290] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:32.015738 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d3d363766546c4a97b0763bad11f71" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:32.016397 28346 leader_election.cc:343] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.021417 28799 tablet_bootstrap.cc:654] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.022264 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 97a9f949999446f09359629fe483bf67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:29 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.023088 29755 raft_consensus.cc:2749] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:32.023227 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet d9c00003300246ada2a356e3e1cf684b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.023448 29755 raft_consensus.cc:493] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.023641 29755 raft_consensus.cc:515] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:32.023699 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 438663f365e34fab87e34f434920cecf consensus state has no leader
I20260618 13:59:32.024344 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet b6dd422c05324c989e29b20ae2afabad with delete_type TABLET_DATA_DELETED (replaced by tablet 97a9f949999446f09359629fe483bf67) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.024447 29755 leader_election.cc:290] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:32.025287 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 233040a16b0d4b61b977ca57c1afec2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.026007 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet aef7c1f111634c98af046e73fa9ffdeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:32.026472 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_efb2b8e451084b7ba0de07e0a6e538c2 [id=333c0fd3837b4d89a9197eb6a826c311], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 9ead386562eb4fd787c7aee776c2194f consensus state has no leader
I20260618 13:59:32.027119 28797 tablet_bootstrap.cc:492] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:32.027318 28797 ts_tablet_manager.cc:1403] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.139s user 0.003s sys 0.001s
I20260618 13:59:32.027396 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 316f470ea82f42fca48e4e96d49c9574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.027597 28798 tablet_bootstrap.cc:492] T 118259402d7d48dda381cdd18eec0f1b P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:32.027741 28798 ts_tablet_manager.cc:1403] T 118259402d7d48dda381cdd18eec0f1b P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.082s user 0.002s sys 0.002s
I20260618 13:59:32.028221 29745 ts_tablet_manager.cc:1929] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:32.028331 29745 log.cc:1199] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/c4cfca9e76e641a7bc0b58b9ee240f36
I20260618 13:59:32.028414 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 2188ebf1727240cd886ba7662f8fcf43 with delete_type TABLET_DATA_DELETED (replaced by tablet c0a42aa779ae452e9420d730bf8d7d55) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.028620 29745 ts_tablet_manager.cc:1950] T c4cfca9e76e641a7bc0b58b9ee240f36 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:32.028825 28406 raft_consensus.cc:2468] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:32.029168 29303 raft_consensus.cc:515] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.029626 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a0039a463ec4a9c82a21a3a69b565fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.029775 29303 leader_election.cc:290] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.031562 29566 ts_tablet_manager.cc:1929] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:32.031652 29566 log.cc:1199] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/41107eb1d9164a9c98060003b4027412
I20260618 13:59:32.031963 29566 ts_tablet_manager.cc:1950] T 41107eb1d9164a9c98060003b4027412 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:32.035305 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 8fd9fa9675234db2abff694a517c0ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet 538077e459a44ed78c95f2e881b6e0b0) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.035845 29745 tablet_replica.cc:333] T 3dd0eaa64271489d92aa380df877f257 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:32.036088 29745 raft_consensus.cc:2243] T 3dd0eaa64271489d92aa380df877f257 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:32.036229 29745 raft_consensus.cc:2272] T 3dd0eaa64271489d92aa380df877f257 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:32.036219 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet aa93f065b4d64e7b85e30f0690fae541 consensus state has no leader
I20260618 13:59:32.036580 29303 raft_consensus.cc:493] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.036703 29303 raft_consensus.cc:515] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.036947 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet c4cfca9e76e641a7bc0b58b9ee240f36 (table default.Table_0e9f574416d84a0f826fa9a1fcc56210 [id=098fa24bb71e4fe1aecc33aea1275ef0]) successfully deleted
I20260618 13:59:32.037155 29745 ts_tablet_manager.cc:1916] T 3dd0eaa64271489d92aa380df877f257 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:32.037335 29303 leader_election.cc:290] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.040367 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fae928833e44d2ba860d6184d4c0142" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:32.041118 28618 leader_election.cc:343] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.042619 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 092a9aba9c6e48f5be9cf5d55ba93678 with delete_type TABLET_DATA_DELETED (replaced by tablet c7f42997416841a7b2c1ad347d2e9282) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.043329 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet d6871232860b41d489e34d95eb897241 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.043320 29476 fs_manager.cc:714] Time spent opening directory manager: real 0.259s user 0.006s sys 0.000s
I20260618 13:59:32.044072 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet eead369de58f40378254697cbbbc6958 with delete_type TABLET_DATA_DELETED (replaced by tablet d464dd402d2044469e364ed37fdfb39b) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.044914 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet edc72a06ee8e4420b07a17d2feb3fcc2 with delete_type TABLET_DATA_DELETED (replaced by tablet aa8cf927bed9437086efd8e0f21ba26a) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.045936 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c75278db2234b11bf95f1cf5cc76edd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.046218 29749 raft_consensus.cc:493] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260618 13:59:32.046367 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 9024952ab8094fcda1329562bbde8ef5 consensus state has no leader
I20260618 13:59:32.046576 29749 raft_consensus.cc:515] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.047523 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet dd2385e6752d49429422fc854539e9c6 with delete_type TABLET_DATA_DELETED (replaced by tablet c38a2156a55e4e5eb91d101e9a61eecd) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.047792 29749 leader_election.cc:290] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:32.047947 28798 raft_consensus.cc:359] T 118259402d7d48dda381cdd18eec0f1b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.048306 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet d502d67826ff4d7a8b0f515e5d8b6d77 with delete_type TABLET_DATA_DELETED (replaced by tablet 657e55fa53c447babfe91b1582806e6b) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.048197 28798 raft_consensus.cc:385] T 118259402d7d48dda381cdd18eec0f1b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.048735 28798 raft_consensus.cc:740] T 118259402d7d48dda381cdd18eec0f1b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.049031 28798 consensus_queue.cc:260] T 118259402d7d48dda381cdd18eec0f1b P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.049582 28798 ts_tablet_manager.cc:1434] T 118259402d7d48dda381cdd18eec0f1b P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.022s user 0.001s sys 0.001s
I20260618 13:59:32.052870 28798 tablet_bootstrap.cc:492] T 6b8f273def3e4b7caa4d624cf87a106c P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:32.053287 28797 raft_consensus.cc:359] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.053700 28797 raft_consensus.cc:385] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.053853 28797 raft_consensus.cc:740] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.054172 28797 consensus_queue.cc:260] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.054999 28797 ts_tablet_manager.cc:1434] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.028s user 0.000s sys 0.002s
I20260618 13:59:32.055711 28797 tablet_bootstrap.cc:492] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:32.057613 28798 tablet_bootstrap.cc:654] T 6b8f273def3e4b7caa4d624cf87a106c P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.058332 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f278d0acbb0c4a0ca61640bc142e6335" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:32.058450 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 751cd9bd961b4f56a588ed02a05f0212 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.058611 28409 raft_consensus.cc:3060] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.060443 28406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd06bb7cffc34412b420d70435fce373" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:32.060623 28406 raft_consensus.cc:2468] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:32.060775 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 7aa9f65d8a5c46fdb745ee17af946dce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.061666 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet d1baf86497d442dbbbddbc3498bebbee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.062490 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet f89c9954d48e407c96c294c26644e25c with delete_type TABLET_DATA_DELETED (replaced by tablet b0d3d363766546c4a97b0763bad11f71) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.063326 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 36b8a46a41e543c58a946b6764618c7d with delete_type TABLET_DATA_DELETED (replaced by tablet f89c9954d48e407c96c294c26644e25c) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.064456 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet bce7bf74b6d24d0bb43808b8a62cf1b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.065357 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet b0285c75c632457383e81805ab036aaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.066395 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b59255b14df4158afe37cb8890e7415 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.067626 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet e95c88e4b0a046a3ad69ee49af47c7c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.068791 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 37a6764a5e024903848561c9935b1a43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.069973 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet fddea8fba4da40d7a5780e6750caf5e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.071482 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 4a01dfb4d59549399c94640f16045555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.072597 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 18d8b609b96a4e81b57f69bff522eeea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.072602 28798 tablet_bootstrap.cc:492] T 6b8f273def3e4b7caa4d624cf87a106c P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:32.073572 28798 ts_tablet_manager.cc:1403] T 6b8f273def3e4b7caa4d624cf87a106c P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.005s
I20260618 13:59:32.074438 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet d667ea5f029f46d89fcb3be03069b97b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.075582 28798 raft_consensus.cc:359] T 6b8f273def3e4b7caa4d624cf87a106c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.075778 28798 raft_consensus.cc:385] T 6b8f273def3e4b7caa4d624cf87a106c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.075867 28798 raft_consensus.cc:740] T 6b8f273def3e4b7caa4d624cf87a106c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.075589 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 69d5e75b4a8247b29d69d07fad5ca5db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.076166 28798 consensus_queue.cc:260] T 6b8f273def3e4b7caa4d624cf87a106c P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:32.076694 28484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.3:35503, user_credentials={real_user=slave}} blocked reactor thread for 110402us
I20260618 13:59:32.076718 28798 ts_tablet_manager.cc:1434] T 6b8f273def3e4b7caa4d624cf87a106c P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:32.078922 28798 tablet_bootstrap.cc:492] T ece7909d056c4ba8a5dca6423f0c1781 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:32.079927 29018 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_a136205448ca4d778d01b1456898276f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:32.080600 29018 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a136205448ca4d778d01b1456898276f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:32.080739 28798 tablet_bootstrap.cc:654] T ece7909d056c4ba8a5dca6423f0c1781 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.081945 29020 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_909b56b640ef4b5db58e13916f138fb0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:32.082520 29020 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_909b56b640ef4b5db58e13916f138fb0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:32.084136 29019 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_473a0c23203b404ea8e5ee0aed201e7b" } modify_external_catalogs: true
I20260618 13:59:32.084348 29019 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_473a0c23203b404ea8e5ee0aed201e7b" } modify_external_catalogs: true
I20260618 13:59:32.090895 28798 tablet_bootstrap.cc:492] T ece7909d056c4ba8a5dca6423f0c1781 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:32.091890 28798 ts_tablet_manager.cc:1403] T ece7909d056c4ba8a5dca6423f0c1781 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.005s
I20260618 13:59:32.093415 28798 raft_consensus.cc:359] T ece7909d056c4ba8a5dca6423f0c1781 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.093619 28798 raft_consensus.cc:385] T ece7909d056c4ba8a5dca6423f0c1781 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.093727 28798 raft_consensus.cc:740] T ece7909d056c4ba8a5dca6423f0c1781 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.093873 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet fe750fc5d998448caeab0a6c808dfba7 with delete_type TABLET_DATA_DELETED (replaced by tablet 438663f365e34fab87e34f434920cecf) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.094064 28798 consensus_queue.cc:260] T ece7909d056c4ba8a5dca6423f0c1781 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.095801 28798 ts_tablet_manager.cc:1434] T ece7909d056c4ba8a5dca6423f0c1781 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 13:59:32.096863 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.097002 28798 tablet_bootstrap.cc:492] T c0a42aa779ae452e9420d730bf8d7d55 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:32.097335 28406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283523e5b22843ba935eabe6283d3a99" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:32.097949 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet b51d0851efb34eaa9e9e4be6b292ef62 with delete_type TABLET_DATA_DELETED (replaced by tablet f9f7359c4a144ca89fe3e51d1365a425) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.099135 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet d4667fe48f0f4eff9c6c8baabdab58d4 with delete_type TABLET_DATA_DELETED (replaced by tablet a662bb37fb7d4df49e61e524808b67a8) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.099275 28798 tablet_bootstrap.cc:654] T c0a42aa779ae452e9420d730bf8d7d55 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.099550 28406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0285c75c632457383e81805ab036aaf" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:32.100744 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet d7a97176f72349cda221baaea61d18ad with delete_type TABLET_DATA_DELETED (replaced by tablet 72fd63e8245d4042b091593881ba11b6) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:32.101094 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet 15180aefba0544329daee74308ef5010 (table default.Table_97b6bb39b3bf4612be391b9dd78d1646 [id=42a2a24590834abc9537669aaf223a7c], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 15180aefba0544329daee74308ef5010 consensus state has no leader
I20260618 13:59:32.101734 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 516421cd843b44769efa2b555697c065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.102257 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d3d363766546c4a97b0763bad11f71" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:32.103026 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f5a334d18ba4c06bb358fa86d8af80d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.104372 29756 raft_consensus.cc:493] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.104609 29756 raft_consensus.cc:515] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:32.104982 28617 leader_election.cc:304] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:32.105468 29756 leader_election.cc:290] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:32.107400 29718 raft_consensus.cc:2804] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:32.107524 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet b16f7d3c7837498e996bba1818110642 with delete_type TABLET_DATA_DELETED (replaced by tablet d1baf86497d442dbbbddbc3498bebbee) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.107826 29718 raft_consensus.cc:697] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:32.108043 28549 raft_consensus.cc:3060] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.108183 29718 consensus_queue.cc:237] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.108626 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet c84189a4b1194c87bbdf81f6e0ed56e4 with delete_type TABLET_DATA_DELETED (replaced by tablet be00bfa9bc1a4b83b18bfad05ba1495e) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:32.109966 28657 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43578: Illegal state: replica d6a169b16ced46ca9cfbc631709bb257 is not leader of this config: current role FOLLOWER
W20260618 13:59:32.110514 28989 catalog_manager.cc:5106] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): alter failed for tablet dc36fbea853a4d4e9f61909200239aed (table default.Table_dbc6162c06f74e71b792bd1f7b2388fe [id=49576dabdecf41aca1b15ff3fc0c113c]): Illegal state: replica d6a169b16ced46ca9cfbc631709bb257 is not leader of this config: current role FOLLOWER
W20260618 13:59:32.111374 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 316110bd2d024c7599fa2933d87e559e consensus state has no leader
I20260618 13:59:32.112323 28548 raft_consensus.cc:3060] T ece7909d056c4ba8a5dca6423f0c1781 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.112352 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet c557afc3abfa4ef188bf248e89f49d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:32.112869 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 4bc87770baa34842a25ed97a19354446 consensus state has no leader
I20260618 13:59:32.113657 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 4c2b924a096b479cad31c29bf7c99172 with delete_type TABLET_DATA_DELETED (replaced by tablet ab521028ac1a4d0da013112d76bb169f) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.114328 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet da4d4f03911f4ac099ecc2fa1d8af494 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.115000 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet aa8cf927bed9437086efd8e0f21ba26a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.116112 29768 tablet.cc:1724] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:32.116547 28547 raft_consensus.cc:3060] T 6b8f273def3e4b7caa4d624cf87a106c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.116631 29522 raft_consensus.cc:493] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.116844 29522 raft_consensus.cc:515] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.117403 29522 leader_election.cc:290] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.119776 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 11) took 1751 ms. Trace:
I20260618 13:59:32.119856 29566 rpcz_store.cc:276] 0618 13:59:30.368746 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:30.368790 (+ 44us) service_pool.cc:224] Handling call
0618 13:59:31.835442 (+1466652us) ts_tablet_manager.cc:1151] Deleting tablet 41107eb1d9164a9c98060003b4027412
0618 13:59:31.835451 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:31.953062 (+117611us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:32.031505 (+ 78443us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:32.119761 (+ 88256us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1466471}
I20260618 13:59:32.120143 29566 tablet_replica.cc:333] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:32.120328 29566 raft_consensus.cc:2243] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:32.120257 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 41107eb1d9164a9c98060003b4027412 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]) successfully deleted
I20260618 13:59:32.121318 29566 raft_consensus.cc:2272] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:32.121666 29522 raft_consensus.cc:493] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.121817 29522 raft_consensus.cc:515] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:32.122121 29770 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 13:59:32.122628 29522 leader_election.cc:290] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:32.123255 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet ece7909d056c4ba8a5dca6423f0c1781 (table default.Table_26e7d8c4f91144268a2a43e83992667d [id=803d5220ce1a4dcfaf27d93c574be2ac], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet ece7909d056c4ba8a5dca6423f0c1781 consensus state has no leader
I20260618 13:59:32.123492 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea4bc13eabc4dd98b4f96723498014c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:32.123653 28681 raft_consensus.cc:2468] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
W20260618 13:59:32.123639 28346 leader_election.cc:343] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.123746 29566 ts_tablet_manager.cc:1916] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:32.123759 28346 leader_election.cc:304] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.124107 28409 raft_consensus.cc:2468] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:32.124521 29640 raft_consensus.cc:515] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.125008 29640 leader_election.cc:290] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.125241 29719 raft_consensus.cc:515] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.125710 29719 leader_election.cc:290] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.127034 28346 leader_election.cc:304] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:32.127290 29679 raft_consensus.cc:2749] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:32.127440 29679 raft_consensus.cc:2804] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:32.127496 29679 raft_consensus.cc:493] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:32.127516 29489 raft_consensus.cc:493] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.127578 29679 raft_consensus.cc:3060] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.127622 29489 raft_consensus.cc:515] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.128114 29489 leader_election.cc:290] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.128684 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ed88c142d7d40cda6392c06bc0486ca" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:32.129150 28346 leader_election.cc:343] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.129558 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet bc0adedecee4405e86b0a37b9e62e352 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.130669 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 7d4144b1b5ea4e968672616bc22bead8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.131508 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet c7f42997416841a7b2c1ad347d2e9282 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.131629 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09cbe2337dd14da1b8e0629ded7e7ad0" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:32.132298 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea4bc13eabc4dd98b4f96723498014c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:32.132917 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ed88c142d7d40cda6392c06bc0486ca" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:32.133868 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 with delete_type TABLET_DATA_DELETED (replaced by tablet cea4bc13eabc4dd98b4f96723498014c) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:32.134235 28483 leader_election.cc:343] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: dca54111fea047d9ae0fabe376d89382
I20260618 13:59:32.134918 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fae928833e44d2ba860d6184d4c0142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:32.135026 28346 leader_election.cc:343] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.136032 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 827cc0987bd541759e334fb620f04ed5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.137480 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 591465e2b57247a2b224cc8b09225811 with delete_type TABLET_DATA_DELETED (replaced by tablet d4667fe48f0f4eff9c6c8baabdab58d4) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.138521 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 864f2602840c40fc8606b3784e671cba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:32.139377 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet fe1139c409e84070a2f496a85545e8d5 consensus state has no leader
I20260618 13:59:32.140566 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f3d2f236fd44ab8af0524ea282cfa94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.141387 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet c552d1316e3340f4bcfa39618ae309b8 with delete_type TABLET_DATA_DELETED (replaced by tablet fe750fc5d998448caeab0a6c808dfba7) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.144443 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 75824a5d2f7e47838b60e6a2b785c6b2 with delete_type TABLET_DATA_DELETED (replaced by tablet d4539f767c054e9e8ebc1f3f56108e7e) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.145277 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 02b5cc08ca0c4e719e0414b345211ba5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:32.145925 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet 39b06ae7444a48dbbec9e096fd7e4052 (table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 39b06ae7444a48dbbec9e096fd7e4052 consensus state has no leader
I20260618 13:59:32.146682 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 6eea4d59935d40aa97dc10ec7469aa56 with delete_type TABLET_DATA_DELETED (replaced by tablet 4bc87770baa34842a25ed97a19354446) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.148550 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 0463fa2e30554abcbfc8f37b702e81d4 with delete_type TABLET_DATA_DELETED (replaced by tablet fe1139c409e84070a2f496a85545e8d5) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:32.148942 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet f278d0acbb0c4a0ca61640bc142e6335 (table default.Table_8783a8d8c526428281436b62b2581a33 [id=0d83fc68255c458587390c58b12e00d7], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet f278d0acbb0c4a0ca61640bc142e6335 consensus state has no leader
I20260618 13:59:32.116571 28617 leader_election.cc:304] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:32.116926 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283523e5b22843ba935eabe6283d3a99" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:32.117396 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0285c75c632457383e81805ab036aaf" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:32.149564 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet e6de1293572f4e75af3417f281b9a142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.117815 29644 consensus_queue.cc:1048] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:32.149899 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet cea4bc13eabc4dd98b4f96723498014c (table default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet cea4bc13eabc4dd98b4f96723498014c consensus state has no leader
I20260618 13:59:32.150040 29640 raft_consensus.cc:2804] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260618 13:59:32.150082 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 957670718c8c4b8cb87115cc4a2fa861 consensus state has no leader
I20260618 13:59:32.119271 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09cbe2337dd14da1b8e0629ded7e7ad0" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:32.150146 29640 raft_consensus.cc:493] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:32.150269 29640 raft_consensus.cc:3060] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.150589 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 04dac931e62a448d99633878eb70b92f with delete_type TABLET_DATA_DELETED (replaced by tablet 15483938a06e459789f87c6bb456bd15) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:32.150576 28346 leader_election.cc:343] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.150702 28346 leader_election.cc:304] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.151846 29522 raft_consensus.cc:2749] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:32.152614 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet be59f988f0e34b9480f859cf88b9c223 with delete_type TABLET_DATA_DELETED (replaced by tablet b968a8d4014341ca9c5964263721ac79) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:32.153236 28662 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43578: Illegal state: replica d6a169b16ced46ca9cfbc631709bb257 is not leader of this config: current role FOLLOWER
I20260618 13:59:32.153802 29756 raft_consensus.cc:493] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260618 13:59:32.154140 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet be00bfa9bc1a4b83b18bfad05ba1495e consensus state has no leader
I20260618 13:59:32.154277 29756 raft_consensus.cc:515] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:32.157294 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet b0d3d363766546c4a97b0763bad11f71 consensus state has no leader
I20260618 13:59:32.158120 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet c3d523419ff244f085d6eac1a3231973 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.158941 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet 015494374645469785f1f978eaf0e9a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 2e493f647a344cd581515a77230d765f) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.159754 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet ac58e939730c4af681c2f08bcfbb6602 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:32.160293 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 09cbe2337dd14da1b8e0629ded7e7ad0 consensus state has no leader
I20260618 13:59:32.161373 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet 7572d17724d249739be1a1cb9468f909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:27 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.160794 28617 leader_election.cc:304] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:32.162128 29644 raft_consensus.cc:2804] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:32.162294 29644 raft_consensus.cc:697] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:32.162596 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet ab521028ac1a4d0da013112d76bb169f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.162599 29644 consensus_queue.cc:237] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:32.164330 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 538077e459a44ed78c95f2e881b6e0b0 consensus state has no leader
W20260618 13:59:32.165696 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 1d8b3cfac0f648cd8b1c51102132c676 consensus state has no leader
I20260618 13:59:32.166066 28799 tablet_bootstrap.cc:492] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:32.166220 28799 ts_tablet_manager.cc:1403] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.160s user 0.022s sys 0.038s
I20260618 13:59:32.166285 28797 tablet_bootstrap.cc:700] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257: Time spent opening tablet: real 0.109s user 0.000s sys 0.001s
W20260618 13:59:32.166620 28346 leader_election.cc:343] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.166958 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e25027f423144b088dad955c7282921 with delete_type TABLET_DATA_DELETED (replaced by tablet a8da96fc621c47d4a444b5a39259f85d) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.167133 28799 raft_consensus.cc:359] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:32.167258 28346 leader_election.cc:343] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.167351 28346 leader_election.cc:304] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.167305 28799 raft_consensus.cc:385] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.167761 28799 raft_consensus.cc:740] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.167791 29522 raft_consensus.cc:2749] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:32.167928 29011 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet c5d046e0421142cd84c5b12d6a434f53 consensus state has no leader
I20260618 13:59:32.167969 28799 consensus_queue.cc:260] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.168195 28797 tablet_bootstrap.cc:654] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.168478 28799 ts_tablet_manager.cc:1434] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:32.169742 28799 tablet_bootstrap.cc:492] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:32.171340 28799 tablet_bootstrap.cc:654] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:32.171592 29011 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.3:42065 (request call id 141) took 1460 ms (client timeout 1000 ms). Trace:
W20260618 13:59:32.171703 29011 rpcz_store.cc:269] 0618 13:59:30.711503 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:30.790844 (+ 79341us) service_pool.cc:224] Handling call
0618 13:59:32.171553 (+1380709us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:31.661108 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:31.661253 (+ 145us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:31.661260 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:31.661262 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:31.661479 (+ 217us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:31.661493 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:31.661498 (+ 5us) tablet.cc:685] Acquiring locks for 6 operations
0618 13:59:31.661628 (+ 130us) tablet.cc:701] Row locks acquired
0618 13:59:31.661630 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:31.661667 (+ 37us) write_op.cc:270] Start()
0618 13:59:31.661702 (+ 35us) write_op.cc:276] Timestamp: P: 1781791171661649 usec, L: 0
0618 13:59:31.661706 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:31.661882 (+ 176us) log.cc:844] Serialized 2416 byte log entry
0618 13:59:31.666159 (+ 4277us) op_driver.cc:464] REPLICATION: finished
0618 13:59:31.666303 (+ 144us) write_op.cc:301] APPLY: starting
0618 13:59:31.666320 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:31.666376 (+ 56us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:31.666408 (+ 32us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:31.667226 (+ 818us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:31.667284 (+ 58us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0618 13:59:31.667296 (+ 12us) write_op.cc:312] APPLY: finished
0618 13:59:31.667427 (+ 131us) log.cc:844] Serialized 53 byte log entry
0618 13:59:31.667505 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:31.667539 (+ 34us) write_op.cc:454] Released schema lock
0618 13:59:31.667550 (+ 11us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:31.667659 (+ 109us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:31.667665 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3504,"reported_tablets":171,"child_traces":[["op",{"apply.queue_time_us":36,"apply.run_cpu_time_us":1335,"apply.run_wall_time_us":1334,"num_ops":6,"prepare.queue_time_us":91,"prepare.run_cpu_time_us":894,"prepare.run_wall_time_us":894,"raft.queue_time_us":833,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":966,"replication_time_us":4424}]]}
I20260618 13:59:32.173751 29756 leader_election.cc:290] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:32.176712 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd06bb7cffc34412b420d70435fce373" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:32.176769 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f278d0acbb0c4a0ca61640bc142e6335" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:32.179456 28546 raft_consensus.cc:3060] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.180480 28543 raft_consensus.cc:3060] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.180744 28549 raft_consensus.cc:3060] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.181818 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aef7c1f111634c98af046e73fa9ffdeb" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:32.182008 28404 raft_consensus.cc:3060] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.182976 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba3d38d32c7c4721bad35f0da7a9ebe5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:32.183136 28409 raft_consensus.cc:3060] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.183202 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aef7c1f111634c98af046e73fa9ffdeb" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:32.183382 28799 tablet_bootstrap.cc:492] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:32.183497 28799 ts_tablet_manager.cc:1403] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.000s
I20260618 13:59:32.183777 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba3d38d32c7c4721bad35f0da7a9ebe5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:32.184242 28799 raft_consensus.cc:359] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.184386 28799 raft_consensus.cc:385] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.184443 28799 raft_consensus.cc:740] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.184674 28799 consensus_queue.cc:260] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.185168 28799 ts_tablet_manager.cc:1434] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:32.186442 28799 tablet_bootstrap.cc:492] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:32.189069 28799 tablet_bootstrap.cc:654] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.202030 29522 raft_consensus.cc:493] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.202229 29522 raft_consensus.cc:515] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.203040 29522 leader_election.cc:290] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:32.204505 29011 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_72b47dac4bd24e24a0eae1a3de512e9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:32.205281 29011 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_72b47dac4bd24e24a0eae1a3de512e9f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:32.212661 28484 leader_election.cc:304] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.213040 28544 raft_consensus.cc:3060] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.218329 29745 ts_tablet_manager.cc:1929] T 3dd0eaa64271489d92aa380df877f257 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:32.218487 29745 log.cc:1199] T 3dd0eaa64271489d92aa380df877f257 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/3dd0eaa64271489d92aa380df877f257
I20260618 13:59:32.218780 29745 ts_tablet_manager.cc:1950] T 3dd0eaa64271489d92aa380df877f257 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:32.219132 29679 raft_consensus.cc:515] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:32.219746 29679 leader_election.cc:290] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:32.222677 28404 raft_consensus.cc:2468] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:32.224939 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 3dd0eaa64271489d92aa380df877f257 (table default.Table_fd6ef45aeec84b48b8fb7a54db4b4f81 [id=ca38f91f52db4c69a0fb3bc6b8f5cadc]) successfully deleted
W20260618 13:59:32.225294 28988 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet e5f7eaa7d218440ca2c35340c5c87814 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5f7eaa7d218440ca2c35340c5c87814 already in progress: creating tablet
I20260618 13:59:32.237555 29476 fs_manager.cc:730] Time spent opening block manager: real 0.118s user 0.002s sys 0.001s
I20260618 13:59:32.242734 29775 tablet.cc:1724] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:32.249670 28545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d464dd402d2044469e364ed37fdfb39b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:32.249752 28541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea4bc13eabc4dd98b4f96723498014c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:32.249763 28483 leader_election.cc:343] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: ba0d05ea44e1473fb4b24f61eab096e4
I20260618 13:59:32.250630 29756 raft_consensus.cc:2749] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:32.250649 28346 leader_election.cc:343] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:32.251159 28346 leader_election.cc:343] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:32.251446 28983 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 957670718c8c4b8cb87115cc4a2fa861 consensus state has no leader
I20260618 13:59:32.255221 29018 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 51) took 1624 ms. Trace:
I20260618 13:59:32.255414 29018 rpcz_store.cc:276] 0618 13:59:30.630633 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:31.973159 (+1342526us) service_pool.cc:224] Handling call
0618 13:59:32.079856 (+106697us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb46846c3 00007fcbab0e07c6 00007fcbab0e04f5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:32.080711 (+ 855us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:32.081191 (+ 480us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:32.254839 (+173648us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:32.254998 (+ 159us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:32.255154 (+ 156us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:32.206089 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:32.206194 (+ 105us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:32.206199 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:32.206201 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:32.206286 (+ 85us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:32.206294 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:32.206298 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:32.206358 (+ 60us) tablet.cc:701] Row locks acquired
0618 13:59:32.206360 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:32.206387 (+ 27us) write_op.cc:270] Start()
0618 13:59:32.216225 (+ 9838us) write_op.cc:276] Timestamp: P: 1781791172206366 usec, L: 0
0618 13:59:32.216231 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:32.216415 (+ 184us) log.cc:844] Serialized 433 byte log entry
0618 13:59:32.252731 (+ 36316us) op_driver.cc:464] REPLICATION: finished
0618 13:59:32.253371 (+ 640us) write_op.cc:301] APPLY: starting
0618 13:59:32.253395 (+ 24us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:32.253458 (+ 63us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:32.253494 (+ 36us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:32.253657 (+ 163us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:32.253699 (+ 42us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:32.253712 (+ 13us) write_op.cc:312] APPLY: finished
0618 13:59:32.253827 (+ 115us) log.cc:844] Serialized 29 byte log entry
0618 13:59:32.253929 (+ 102us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.253960 (+ 31us) write_op.cc:454] Released schema lock
0618 13:59:32.253972 (+ 12us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:32.254078 (+ 106us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.254084 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":244643712,"child_traces":[["op",{"apply.queue_time_us":556,"apply.run_cpu_time_us":703,"apply.run_wall_time_us":702,"num_ops":2,"prepare.queue_time_us":124396,"prepare.run_cpu_time_us":680,"prepare.run_wall_time_us":10847,"raft.queue_time_us":13418,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":737,"replication_time_us":36446}]]}
W20260618 13:59:32.259683 28484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.3:35503, user_credentials={real_user=slave}} blocked reactor thread for 47061.2us
I20260618 13:59:32.260010 28484 leader_election.cc:304] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.260310 29020 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 52) took 1581 ms. Trace:
I20260618 13:59:32.260470 29020 rpcz_store.cc:276] 0618 13:59:30.679043 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.002315 (+1323272us) service_pool.cc:224] Handling call
0618 13:59:32.081895 (+ 79580us) spinlock_profiling.cc:243] Waited 78.9 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb46846c3 00007fcbab0e07c6 00007fcbab0e04f5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:32.082647 (+ 752us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:32.083205 (+ 558us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:32.260008 (+176803us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:32.260150 (+ 142us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:32.260260 (+ 110us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:32.216979 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:32.217129 (+ 150us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:32.217137 (+ 8us) write_op.cc:435] Acquired schema lock
0618 13:59:32.217141 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:32.217257 (+ 116us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:32.217272 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:32.217279 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:32.217404 (+ 125us) tablet.cc:701] Row locks acquired
0618 13:59:32.217408 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:32.217443 (+ 35us) write_op.cc:270] Start()
0618 13:59:32.217480 (+ 37us) write_op.cc:276] Timestamp: P: 1781791172217417 usec, L: 0
0618 13:59:32.217486 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:32.217689 (+ 203us) log.cc:844] Serialized 433 byte log entry
0618 13:59:32.252821 (+ 35132us) op_driver.cc:464] REPLICATION: finished
0618 13:59:32.254191 (+ 1370us) write_op.cc:301] APPLY: starting
0618 13:59:32.254209 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:32.254254 (+ 45us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:32.254297 (+ 43us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:32.259615 (+ 5318us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:32.259656 (+ 41us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:32.259667 (+ 11us) write_op.cc:312] APPLY: finished
0618 13:59:32.259757 (+ 90us) log.cc:844] Serialized 29 byte log entry
0618 13:59:32.259828 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.259853 (+ 25us) write_op.cc:454] Released schema lock
0618 13:59:32.259862 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":181484928,"child_traces":[["op",{"apply.queue_time_us":1325,"apply.run_cpu_time_us":666,"apply.run_wall_time_us":5744,"num_ops":2,"prepare.queue_time_us":133197,"replication_time_us":35314}]]}
I20260618 13:59:32.261754 29756 raft_consensus.cc:2749] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:32.261626 29020 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_1674937bcc0342af83719008ac10377d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:32.262238 28484 leader_election.cc:343] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 0a51e830e3014726b1d1c2608973f036
W20260618 13:59:32.262300 29020 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1674937bcc0342af83719008ac10377d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:32.267267 29679 raft_consensus.cc:493] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.267467 29679 raft_consensus.cc:515] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.268296 29679 leader_election.cc:290] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.269189 28541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "015494374645469785f1f978eaf0e9a5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:32.269755 28346 leader_election.cc:343] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.262392 28484 leader_election.cc:304] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.270238 29756 raft_consensus.cc:2749] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:32.271483 28484 leader_election.cc:304] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.272807 28484 leader_election.cc:304] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.279052 28798 tablet_bootstrap.cc:492] T c0a42aa779ae452e9420d730bf8d7d55 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:32.279245 28798 ts_tablet_manager.cc:1403] T c0a42aa779ae452e9420d730bf8d7d55 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.183s user 0.006s sys 0.006s
I20260618 13:59:32.170447 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet 283523e5b22843ba935eabe6283d3a99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.181373 29718 raft_consensus.cc:493] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.283537 28484 leader_election.cc:304] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.283479 29718 raft_consensus.cc:515] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.284202 29718 leader_election.cc:290] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.203476 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d464dd402d2044469e364ed37fdfb39b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:32.207326 28657 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43578: Illegal state: replica d6a169b16ced46ca9cfbc631709bb257 is not leader of this config: current role FOLLOWER
I20260618 13:59:32.223066 29640 raft_consensus.cc:515] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.223729 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea4bc13eabc4dd98b4f96723498014c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
W20260618 13:59:32.285003 28346 leader_election.cc:343] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.285084 28680 raft_consensus.cc:3060] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.285135 28346 leader_election.cc:304] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.285460 29640 leader_election.cc:290] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.285579 29679 raft_consensus.cc:2749] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:32.242700 29782 tablet.cc:1724] T 6b8f273def3e4b7caa4d624cf87a106c P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260618 13:59:32.243104 29783 tablet.cc:1724] T 512a9e63ba324a30b8d8c37e585ebdf8 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:32.249790 29644 consensus_queue.cc:1048] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:32.250061 28617 leader_election.cc:304] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:32.269157 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "015494374645469785f1f978eaf0e9a5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:32.289533 28346 leader_election.cc:343] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.289669 28346 leader_election.cc:304] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.290045 29679 raft_consensus.cc:2749] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:32.290485 29644 raft_consensus.cc:2804] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:32.290686 29644 raft_consensus.cc:697] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:32.290982 29644 consensus_queue.cc:237] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.299858 29644 raft_consensus.cc:493] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.300076 29644 raft_consensus.cc:515] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.300941 29644 leader_election.cc:290] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.300968 29522 raft_consensus.cc:493] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.301128 29522 raft_consensus.cc:515] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:32.301908 29522 leader_election.cc:290] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:32.302573 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d014b46a0f448b929cbc9287ead0a7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:32.302748 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b06ae7444a48dbbec9e096fd7e4052" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:32.302922 28408 raft_consensus.cc:2468] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:32.302747 28404 raft_consensus.cc:2468] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:32.304729 28617 leader_election.cc:304] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:32.305044 28617 leader_election.cc:304] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:32.305047 29718 raft_consensus.cc:2804] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:32.305363 29718 raft_consensus.cc:493] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:32.305500 29718 raft_consensus.cc:3060] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.306814 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab521028ac1a4d0da013112d76bb169f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:32.307348 28346 leader_election.cc:343] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.307778 28547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab521028ac1a4d0da013112d76bb169f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:32.308310 29644 raft_consensus.cc:2804] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:32.308439 29644 raft_consensus.cc:493] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:32.308564 29644 raft_consensus.cc:3060] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.308554 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd06bb7cffc34412b420d70435fce373" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:32.308748 28407 raft_consensus.cc:3060] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:32.308951 28346 leader_election.cc:343] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.309056 28346 leader_election.cc:304] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.309422 29679 raft_consensus.cc:2749] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:32.313778 29759 raft_consensus.cc:2749] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:32.314527 29756 raft_consensus.cc:2749] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:32.317687 29790 tablet.cc:1724] T 1f81428f804a485c967026c4527fd250 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260618 13:59:32.319661 28618 leader_election.cc:343] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:32.320029 28618 leader_election.cc:343] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.331911 28409 raft_consensus.cc:2468] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:32.333377 29791 tablet.cc:1724] T a1a06078974e4bfbbc4b949802a39bf1 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:32.334832 28407 raft_consensus.cc:2468] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:32.337293 29790 tablet.cc:1724] T 512a9e63ba324a30b8d8c37e585ebdf8 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:32.338603 28797 tablet_bootstrap.cc:492] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:32.338764 28680 raft_consensus.cc:2468] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:32.343314 28346 leader_election.cc:304] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:32.343843 29679 raft_consensus.cc:2804] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:32.343986 29679 raft_consensus.cc:697] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:32.344301 29679 consensus_queue.cc:237] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:32.345793 28797 ts_tablet_manager.cc:1403] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.290s user 0.001s sys 0.003s
I20260618 13:59:32.351301 29789 tablet.cc:1724] T 6b8f273def3e4b7caa4d624cf87a106c P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260618 13:59:32.237910 29476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data,/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/wal
uuid: "7637285570574a6dbcef1de6f84ecac0"
format_stamp: "Formatted at 2026-06-18 13:59:16 on dist-test-slave-bmq2"
I20260618 13:59:32.355028 29476 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 13:59:32.349810 28617 leader_election.cc:304] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:32.341486 29768 tablet.cc:1724] T 1f81428f804a485c967026c4527fd250 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:32.361865 28617 leader_election.cc:304] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:32.354671 28797 raft_consensus.cc:359] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:32.362371 29607 raft_consensus.cc:2804] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:32.362917 28797 raft_consensus.cc:385] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.363456 28797 raft_consensus.cc:740] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.364037 29607 raft_consensus.cc:697] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:32.364324 29607 consensus_queue.cc:237] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.364171 28797 consensus_queue.cc:260] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:32.367743 28797 ts_tablet_manager.cc:1434] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.022s user 0.002s sys 0.001s
W20260618 13:59:32.373747 28657 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43578: Illegal state: replica d6a169b16ced46ca9cfbc631709bb257 is not leader of this config: current role FOLLOWER
W20260618 13:59:32.378724 28618 leader_election.cc:343] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.362010 29719 raft_consensus.cc:2804] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
W20260618 13:59:32.379093 28618 leader_election.cc:343] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.371762 29749 raft_consensus.cc:493] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.383064 29749 raft_consensus.cc:515] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.383713 29749 leader_election.cc:290] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:32.384507 29011 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 53) took 1705 ms. Trace:
I20260618 13:59:32.384632 29011 rpcz_store.cc:276] 0618 13:59:30.679208 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.173483 (+1494275us) service_pool.cc:224] Handling call
0618 13:59:32.204420 (+ 30937us) spinlock_profiling.cc:243] Waited 30.6 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb46846c3 00007fcbab0e07c6 00007fcbab0e04f5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:32.205430 (+ 1010us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:32.206005 (+ 575us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:32.328823 (+122818us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:32.384339 (+ 55516us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:32.384463 (+ 124us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:32.280234 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:32.280357 (+ 123us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:32.280363 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:32.280367 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:32.280460 (+ 93us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:32.280472 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:32.280477 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:32.280575 (+ 98us) tablet.cc:701] Row locks acquired
0618 13:59:32.280578 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:32.280616 (+ 38us) write_op.cc:270] Start()
0618 13:59:32.280653 (+ 37us) write_op.cc:276] Timestamp: P: 1781791172280582 usec, L: 0
0618 13:59:32.280659 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:32.280841 (+ 182us) log.cc:844] Serialized 433 byte log entry
0618 13:59:32.327442 (+ 46601us) op_driver.cc:464] REPLICATION: finished
0618 13:59:32.328348 (+ 906us) write_op.cc:301] APPLY: starting
0618 13:59:32.328360 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:32.328409 (+ 49us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:32.328432 (+ 23us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:32.328556 (+ 124us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:32.328589 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:32.328596 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:32.328669 (+ 73us) log.cc:844] Serialized 29 byte log entry
0618 13:59:32.328715 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.328730 (+ 15us) write_op.cc:454] Released schema lock
0618 13:59:32.328735 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:32.328813 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.328818 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":70325120,"child_traces":[["op",{"apply.queue_time_us":843,"apply.run_cpu_time_us":458,"apply.run_wall_time_us":458,"num_ops":2,"prepare.queue_time_us":73591,"prepare.run_cpu_time_us":709,"prepare.run_wall_time_us":1117,"raft.queue_time_us":32,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":46751}]]}
I20260618 13:59:32.385371 29011 master_service.cc:960] ReplaceTablet: received request to replace tablet f278d0acbb0c4a0ca61640bc142e6335 from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:32.387461 29020 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 54) took 1701 ms. Trace:
I20260618 13:59:32.387574 29020 rpcz_store.cc:276] 0618 13:59:30.685526 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.261361 (+1575835us) service_pool.cc:224] Handling call
0618 13:59:32.262399 (+ 1038us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:32.267027 (+ 4628us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:32.329303 (+ 62276us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:32.387329 (+ 58026us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:32.387423 (+ 94us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:32.281378 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:32.281477 (+ 99us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:32.281483 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:32.281487 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:32.281578 (+ 91us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:32.281589 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:32.281594 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:32.281665 (+ 71us) tablet.cc:701] Row locks acquired
0618 13:59:32.281667 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:32.281699 (+ 32us) write_op.cc:270] Start()
0618 13:59:32.281726 (+ 27us) write_op.cc:276] Timestamp: P: 1781791172281669 usec, L: 0
0618 13:59:32.281730 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:32.281884 (+ 154us) log.cc:844] Serialized 433 byte log entry
0618 13:59:32.327509 (+ 45625us) op_driver.cc:464] REPLICATION: finished
0618 13:59:32.328888 (+ 1379us) write_op.cc:301] APPLY: starting
0618 13:59:32.328899 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:32.328928 (+ 29us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:32.328958 (+ 30us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:32.329044 (+ 86us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:32.329071 (+ 27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:32.329078 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:32.329153 (+ 75us) log.cc:844] Serialized 29 byte log entry
0618 13:59:32.329194 (+ 41us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.329218 (+ 24us) write_op.cc:454] Released schema lock
0618 13:59:32.329224 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1341,"apply.run_cpu_time_us":403,"apply.run_wall_time_us":402,"num_ops":2,"prepare.queue_time_us":13805,"prepare.run_cpu_time_us":602,"prepare.run_wall_time_us":1039,"raft.queue_time_us":32,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":7,"replication_time_us":45762,"thread_start_us":11281,"threads_started":1}]]}
I20260618 13:59:32.388410 29020 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_49b3aa53eeeb4a208dcec66719481e47"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:32.388999 29020 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_49b3aa53eeeb4a208dcec66719481e47 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:32.379565 29719 raft_consensus.cc:697] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:32.389407 29719 consensus_queue.cc:237] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:32.394640 28483 leader_election.cc:343] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 94365c44b5e24acab5df60c67203794a
I20260618 13:59:32.395321 28548 raft_consensus.cc:3060] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.402668 29787 tablet.cc:1724] T a1a06078974e4bfbbc4b949802a39bf1 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:32.361711 28798 raft_consensus.cc:359] T c0a42aa779ae452e9420d730bf8d7d55 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:32.403854 28798 raft_consensus.cc:385] T c0a42aa779ae452e9420d730bf8d7d55 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.403990 28798 raft_consensus.cc:740] T c0a42aa779ae452e9420d730bf8d7d55 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.404228 28797 tablet_bootstrap.cc:492] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:32.407449 29020 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 56) took 1589 ms. Trace:
I20260618 13:59:32.408004 28596 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:32.407598 29020 rpcz_store.cc:276] 0618 13:59:30.817810 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.388191 (+1570381us) service_pool.cc:224] Handling call
0618 13:59:32.389094 (+ 903us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:32.389557 (+ 463us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:32.407181 (+ 17624us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:32.407296 (+ 115us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:32.407389 (+ 93us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:32.395758 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:32.395862 (+ 104us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:32.395867 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:32.395870 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:32.395942 (+ 72us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:32.395950 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:32.395954 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:32.396026 (+ 72us) tablet.cc:701] Row locks acquired
0618 13:59:32.396029 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:32.396056 (+ 27us) write_op.cc:270] Start()
0618 13:59:32.396081 (+ 25us) write_op.cc:276] Timestamp: P: 1781791172396016 usec, L: 0
0618 13:59:32.396084 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:32.396218 (+ 134us) log.cc:844] Serialized 433 byte log entry
0618 13:59:32.406063 (+ 9845us) op_driver.cc:464] REPLICATION: finished
0618 13:59:32.406393 (+ 330us) write_op.cc:301] APPLY: starting
0618 13:59:32.406407 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:32.406466 (+ 59us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:32.406501 (+ 35us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:32.406670 (+ 169us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:32.406704 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:32.406713 (+ 9us) write_op.cc:312] APPLY: finished
0618 13:59:32.406786 (+ 73us) log.cc:844] Serialized 29 byte log entry
0618 13:59:32.407003 (+ 217us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.407021 (+ 18us) write_op.cc:454] Released schema lock
0618 13:59:32.407027 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:32.407091 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.407095 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":272,"apply.run_cpu_time_us":532,"apply.run_wall_time_us":689,"num_ops":2,"prepare.queue_time_us":5765,"prepare.run_cpu_time_us":538,"prepare.run_wall_time_us":1499,"raft.queue_time_us":786,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":4,"replication_time_us":9948}]]}
I20260618 13:59:32.408435 29020 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_4d777a002dff4c77b022c9d8f05a3804"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:32.409049 29020 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d777a002dff4c77b022c9d8f05a3804 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:32.409446 29011 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet f278d0acbb0c4a0ca61640bc142e6335
I20260618 13:59:32.411204 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet f278d0acbb0c4a0ca61640bc142e6335 with delete_type TABLET_DATA_DELETED (replaced by tablet 72c04f55890648f0a8184f3783243f7c) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.411487 28525 tablet_service.cc:1558] Processing DeleteTablet for tablet f278d0acbb0c4a0ca61640bc142e6335 with delete_type TABLET_DATA_DELETED (replaced by tablet 72c04f55890648f0a8184f3783243f7c) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:32.411739 29745 tablet_replica.cc:333] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:32.411954 29745 raft_consensus.cc:2243] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:32.412119 29745 raft_consensus.cc:2272] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:32.412636 29745 ts_tablet_manager.cc:1916] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:32.415110 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet f278d0acbb0c4a0ca61640bc142e6335 with delete_type TABLET_DATA_DELETED (replaced by tablet 72c04f55890648f0a8184f3783243f7c) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.416631 28545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d014b46a0f448b929cbc9287ead0a7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:32.416718 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet f278d0acbb0c4a0ca61640bc142e6335 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f278d0acbb0c4a0ca61640bc142e6335 already in progress: creating tablet
I20260618 13:59:32.416863 28539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b06ae7444a48dbbec9e096fd7e4052" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:32.417122 29011 catalog_manager.cc:6784] ReplaceTablet: tablet f278d0acbb0c4a0ca61640bc142e6335 deleted and replaced by tablet 72c04f55890648f0a8184f3783243f7c
I20260618 13:59:32.417261 29011 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 55) took 1621 ms. Trace:
I20260618 13:59:32.417138 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd06bb7cffc34412b420d70435fce373" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:32.417758 28539 raft_consensus.cc:3060] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.417346 29011 rpcz_store.cc:276] 0618 13:59:30.795856 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.385306 (+1589450us) service_pool.cc:224] Handling call
0618 13:59:32.417213 (+ 31907us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:32.394925 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:32.395047 (+ 122us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:32.395054 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:32.395057 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:32.395189 (+ 132us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:32.395203 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:32.395208 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:32.395296 (+ 88us) tablet.cc:701] Row locks acquired
0618 13:59:32.395300 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:32.395342 (+ 42us) write_op.cc:270] Start()
0618 13:59:32.395386 (+ 44us) write_op.cc:276] Timestamp: P: 1781791172395297 usec, L: 0
0618 13:59:32.395391 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:32.395619 (+ 228us) log.cc:844] Serialized 625 byte log entry
0618 13:59:32.404922 (+ 9303us) op_driver.cc:464] REPLICATION: finished
0618 13:59:32.405014 (+ 92us) write_op.cc:301] APPLY: starting
0618 13:59:32.405034 (+ 20us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:32.405107 (+ 73us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:32.405137 (+ 30us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:32.405248 (+ 111us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:32.405332 (+ 84us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:32.405341 (+ 9us) write_op.cc:312] APPLY: finished
0618 13:59:32.405420 (+ 79us) log.cc:844] Serialized 29 byte log entry
0618 13:59:32.405481 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.405499 (+ 18us) write_op.cc:454] Released schema lock
0618 13:59:32.405505 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":37,"apply.run_cpu_time_us":563,"apply.run_wall_time_us":563,"num_ops":2,"prepare.queue_time_us":8415,"prepare.run_cpu_time_us":766,"prepare.run_wall_time_us":804,"raft.queue_time_us":1239,"raft.run_cpu_time_us":698,"raft.run_wall_time_us":697,"replication_time_us":9496}]]}
I20260618 13:59:32.419057 29011 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_19806c50abb24f379c7afe2107b88b7f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:32.420248 29011 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_19806c50abb24f379c7afe2107b88b7f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:32.434738 29566 ts_tablet_manager.cc:1929] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:32.434883 29566 log.cc:1199] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/8b88ed7cf79c4929a8704f66691cbed4
I20260618 13:59:32.434813 29775 tablet.cc:1724] T 9b75ece59d924e3db49e002bf0faa89c P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:32.435350 29566 ts_tablet_manager.cc:1950] T 8b88ed7cf79c4929a8704f66691cbed4 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:32.434275 29718 raft_consensus.cc:515] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.441665 29718 leader_election.cc:290] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.433827 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 52006500f117417b8a2c979d4aeb077c (DEFAULT_TABLE table=default.Table_3a7cb505ade04e6fa187bf8690075057 [id=6b334b976146443b90db6e4e53492c47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.445235 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b06ae7444a48dbbec9e096fd7e4052" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:32.445463 28403 raft_consensus.cc:3060] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.447184 28657 tablet_service.cc:1511] Processing CreateTablet for tablet 52006500f117417b8a2c979d4aeb077c (DEFAULT_TABLE table=default.Table_3a7cb505ade04e6fa187bf8690075057 [id=6b334b976146443b90db6e4e53492c47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.447890 28525 tablet_service.cc:1511] Processing CreateTablet for tablet 52006500f117417b8a2c979d4aeb077c (DEFAULT_TABLE table=default.Table_3a7cb505ade04e6fa187bf8690075057 [id=6b334b976146443b90db6e4e53492c47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.448335 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52006500f117417b8a2c979d4aeb077c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.452445 28662 tablet_service.cc:1511] Processing CreateTablet for tablet c5836982752b4209b72a372c1dfeb71c (DEFAULT_TABLE table=default.Table_73f07f55ca3145b89d0559c0ff579856 [id=a6a5da4d73a042ffb816e90216c97166]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.453249 28512 tablet_service.cc:1511] Processing CreateTablet for tablet c5836982752b4209b72a372c1dfeb71c (DEFAULT_TABLE table=default.Table_73f07f55ca3145b89d0559c0ff579856 [id=a6a5da4d73a042ffb816e90216c97166]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.454493 28512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5836982752b4209b72a372c1dfeb71c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.457597 28515 tablet_service.cc:1511] Processing CreateTablet for tablet c9ceb3af08fa4618b81cf6f89d7bd0df (DEFAULT_TABLE table=default.Table_5f54daf83a3347cc885616848fee0f43 [id=10a4f5ef579746b68d8dd37d83f14c3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.457962 28515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9ceb3af08fa4618b81cf6f89d7bd0df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.458880 28373 tablet_service.cc:1511] Processing CreateTablet for tablet c5836982752b4209b72a372c1dfeb71c (DEFAULT_TABLE table=default.Table_73f07f55ca3145b89d0559c0ff579856 [id=a6a5da4d73a042ffb816e90216c97166]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.465582 29020 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 57) took 1627 ms. Trace:
I20260618 13:59:32.465737 29020 rpcz_store.cc:276] 0618 13:59:30.837887 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.408239 (+1570352us) service_pool.cc:224] Handling call
0618 13:59:32.409144 (+ 905us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:32.409860 (+ 716us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:32.465334 (+ 55474us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:32.465446 (+ 112us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:32.465538 (+ 92us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:32.453611 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:32.453721 (+ 110us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:32.453728 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:32.453732 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:32.460495 (+ 6763us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:32.460520 (+ 25us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:32.460526 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:32.460624 (+ 98us) tablet.cc:701] Row locks acquired
0618 13:59:32.460627 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:32.460665 (+ 38us) write_op.cc:270] Start()
0618 13:59:32.460713 (+ 48us) write_op.cc:276] Timestamp: P: 1781791172460615 usec, L: 0
0618 13:59:32.460716 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:32.460918 (+ 202us) log.cc:844] Serialized 433 byte log entry
0618 13:59:32.463780 (+ 2862us) op_driver.cc:464] REPLICATION: finished
0618 13:59:32.464214 (+ 434us) write_op.cc:301] APPLY: starting
0618 13:59:32.464230 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:32.464276 (+ 46us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:32.464308 (+ 32us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:32.464423 (+ 115us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:32.464459 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:32.464466 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:32.464536 (+ 70us) log.cc:844] Serialized 29 byte log entry
0618 13:59:32.464614 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.464632 (+ 18us) write_op.cc:454] Released schema lock
0618 13:59:32.464638 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":378,"apply.run_cpu_time_us":514,"apply.run_wall_time_us":512,"num_ops":2,"prepare.queue_time_us":43059,"raft.queue_time_us":121,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":215,"replication_time_us":3036}]]}
I20260618 13:59:32.466377 28385 tablet_service.cc:1511] Processing CreateTablet for tablet c9ceb3af08fa4618b81cf6f89d7bd0df (DEFAULT_TABLE table=default.Table_5f54daf83a3347cc885616848fee0f43 [id=10a4f5ef579746b68d8dd37d83f14c3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.466949 29020 master_service.cc:960] ReplaceTablet: received request to replace tablet d3dbf13544ce408ab97834aec5862390 from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:32.468012 29807 tablet.cc:1724] T 1f81428f804a485c967026c4527fd250 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:32.468458 28653 tablet_service.cc:1511] Processing CreateTablet for tablet c9ceb3af08fa4618b81cf6f89d7bd0df (DEFAULT_TABLE table=default.Table_5f54daf83a3347cc885616848fee0f43 [id=10a4f5ef579746b68d8dd37d83f14c3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.470587 28545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b06ae7444a48dbbec9e096fd7e4052" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:32.475018 29303 consensus_queue.cc:1048] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:32.482684 28484 leader_election.cc:343] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: dca54111fea047d9ae0fabe376d89382
I20260618 13:59:32.485527 29011 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 58) took 1402 ms. Trace:
I20260618 13:59:32.485636 29011 rpcz_store.cc:276] 0618 13:59:31.083513 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.418807 (+1335294us) service_pool.cc:224] Handling call
0618 13:59:32.420425 (+ 1618us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:32.421524 (+ 1099us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:32.484971 (+ 63447us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:32.485366 (+ 395us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:32.485466 (+ 100us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:32.477179 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:32.477323 (+ 144us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:32.477329 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:32.477332 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:32.477428 (+ 96us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:32.477442 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:32.477446 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:32.477524 (+ 78us) tablet.cc:701] Row locks acquired
0618 13:59:32.477526 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:32.477557 (+ 31us) write_op.cc:270] Start()
0618 13:59:32.477589 (+ 32us) write_op.cc:276] Timestamp: P: 1781791172477506 usec, L: 0
0618 13:59:32.477593 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:32.477792 (+ 199us) log.cc:844] Serialized 433 byte log entry
0618 13:59:32.481959 (+ 4167us) op_driver.cc:464] REPLICATION: finished
0618 13:59:32.482057 (+ 98us) write_op.cc:301] APPLY: starting
0618 13:59:32.482074 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:32.482119 (+ 45us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:32.482149 (+ 30us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:32.482271 (+ 122us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:32.482307 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:32.482316 (+ 9us) write_op.cc:312] APPLY: finished
0618 13:59:32.482392 (+ 76us) log.cc:844] Serialized 29 byte log entry
0618 13:59:32.482479 (+ 87us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.482499 (+ 20us) write_op.cc:454] Released schema lock
0618 13:59:32.482507 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":35,"apply.run_cpu_time_us":514,"apply.run_wall_time_us":514,"num_ops":2,"prepare.queue_time_us":55149,"raft.queue_time_us":50,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":208,"replication_time_us":4340}]]}
W20260618 13:59:32.486913 28618 leader_election.cc:343] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:32.487389 28618 leader_election.cc:343] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:32.488096 28618 leader_election.cc:343] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.488624 29011 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_9c526031302044508f50fead286b4465" } new_table_name: "default.Table_599a2901aeaf4edc9cc2f967fb5fcbae" modify_external_catalogs: true
I20260618 13:59:32.486907 29808 tablet.cc:1724] T a1a06078974e4bfbbc4b949802a39bf1 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:32.490135 28484 leader_election.cc:304] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.491638 29749 raft_consensus.cc:2749] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:32.494609 28484 leader_election.cc:343] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: ba0d05ea44e1473fb4b24f61eab096e4
I20260618 13:59:32.495698 29679 raft_consensus.cc:493] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.495868 29679 raft_consensus.cc:515] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.496490 29679 leader_election.cc:290] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.496673 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 12) took 2127 ms. Trace:
I20260618 13:59:32.496794 29566 rpcz_store.cc:276] 0618 13:59:30.369256 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:30.369301 (+ 45us) service_pool.cc:224] Handling call
0618 13:59:32.120103 (+1750802us) ts_tablet_manager.cc:1151] Deleting tablet 8b88ed7cf79c4929a8704f66691cbed4
0618 13:59:32.120111 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:32.227350 (+107239us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:32.434622 (+207272us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:32.496636 (+ 62014us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1750660}
I20260618 13:59:32.497197 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 14) took 2097 ms. Trace:
I20260618 13:59:32.497313 29566 rpcz_store.cc:276] 0618 13:59:30.399655 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:30.399712 (+ 57us) service_pool.cc:224] Handling call
0618 13:59:32.497056 (+2097344us) ts_tablet_manager.cc:1151] Deleting tablet f3ba9186b3ff47d5a2b158fef619686e
0618 13:59:32.497065 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:32.497166 (+ 101us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2097097}
I20260618 13:59:32.497524 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572d17724d249739be1a1cb9468f909" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:32.497656 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 15) took 2055 ms. Trace:
I20260618 13:59:32.497766 29566 rpcz_store.cc:276] 0618 13:59:30.442038 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:30.442092 (+ 54us) service_pool.cc:224] Handling call
0618 13:59:32.497543 (+2055451us) ts_tablet_manager.cc:1151] Deleting tablet ca2ee746c5024cfc9b31c470f9616e7d
0618 13:59:32.497551 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:32.497622 (+ 71us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2034000}
I20260618 13:59:32.498034 29566 tablet_replica.cc:333] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
W20260618 13:59:32.498046 28346 leader_election.cc:343] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.498232 29566 raft_consensus.cc:2243] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:32.498556 29566 raft_consensus.cc:2272] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:32.499101 29566 ts_tablet_manager.cc:1916] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:32.501662 28797 tablet_bootstrap.cc:654] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.505553 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 8b88ed7cf79c4929a8704f66691cbed4 (table default.Table_1333d91d141a46acac33460026215195 [id=232810f09f9e41269d67ccd55527ab8a]) successfully deleted
W20260618 13:59:32.505926 28989 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet f3ba9186b3ff47d5a2b158fef619686e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f3ba9186b3ff47d5a2b158fef619686e already in progress: creating tablet
W20260618 13:59:32.506219 28989 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet ca2ee746c5024cfc9b31c470f9616e7d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ca2ee746c5024cfc9b31c470f9616e7d already in progress: creating tablet
I20260618 13:59:32.507105 29476 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260618 13:59:32.507277 28484 leader_election.cc:304] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.507828 29749 raft_consensus.cc:2749] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:32.509073 28484 leader_election.cc:343] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 94365c44b5e24acab5df60c67203794a
I20260618 13:59:32.509192 28484 leader_election.cc:304] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.509871 29749 raft_consensus.cc:2749] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260618 13:59:32.511389 28618 leader_election.cc:343] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.490134 28798 consensus_queue.cc:260] T c0a42aa779ae452e9420d730bf8d7d55 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:32.501320 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572d17724d249739be1a1cb9468f909" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:32.518148 28798 ts_tablet_manager.cc:1434] T c0a42aa779ae452e9420d730bf8d7d55 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.239s user 0.003s sys 0.000s
W20260618 13:59:32.518222 28346 leader_election.cc:343] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.518432 28346 leader_election.cc:304] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.518508 28798 tablet_bootstrap.cc:492] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:32.518890 29679 raft_consensus.cc:2749] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:32.520299 28798 tablet_bootstrap.cc:654] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.523947 29020 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet d3dbf13544ce408ab97834aec5862390
I20260618 13:59:32.524883 28372 tablet_service.cc:1558] Processing DeleteTablet for tablet d3dbf13544ce408ab97834aec5862390 with delete_type TABLET_DATA_DELETED (replaced by tablet ba18e879631947f88bd266681746ec82) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.525293 29020 catalog_manager.cc:6784] ReplaceTablet: tablet d3dbf13544ce408ab97834aec5862390 deleted and replaced by tablet ba18e879631947f88bd266681746ec82
I20260618 13:59:32.525480 29020 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 59) took 1416 ms. Trace:
I20260618 13:59:32.526113 29749 raft_consensus.cc:493] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:32.526278 29749 raft_consensus.cc:515] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.526821 29749 leader_election.cc:290] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:32.528079 28483 leader_election.cc:343] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: c84189a4b1194c87bbdf81f6e0ed56e4
I20260618 13:59:32.525614 29020 rpcz_store.cc:276] 0618 13:59:31.108749 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.466866 (+1358117us) service_pool.cc:224] Handling call
0618 13:59:32.478356 (+ 11490us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb469154b 00007fcbab0ed107 00007fcbab0ece35 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:32.525423 (+ 47067us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:32.487231 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:32.487334 (+ 103us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:32.487339 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:32.487341 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:32.487445 (+ 104us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:32.487459 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:32.487466 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:32.487545 (+ 79us) tablet.cc:701] Row locks acquired
0618 13:59:32.487549 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:32.487590 (+ 41us) write_op.cc:270] Start()
0618 13:59:32.487627 (+ 37us) write_op.cc:276] Timestamp: P: 1781791172487535 usec, L: 0
0618 13:59:32.487633 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:32.500168 (+ 12535us) log.cc:844] Serialized 659 byte log entry
0618 13:59:32.500977 (+ 809us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fcbac75ef22 00007fcbadb40305 00007fcbadb3ee29 00007fcbadb47250 00007fcbadb4705d 0000000000aa681c 00007fcba07299ff 00007fcba072cd1b 00007fcba072cb98 0000000000aa681c
0618 13:59:32.522372 (+ 21395us) op_driver.cc:464] REPLICATION: finished
0618 13:59:32.522497 (+ 125us) write_op.cc:301] APPLY: starting
0618 13:59:32.522515 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:32.522561 (+ 46us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:32.522564 (+ 3us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:32.522687 (+ 123us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:32.522745 (+ 58us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:32.522752 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:32.522830 (+ 78us) log.cc:844] Serialized 29 byte log entry
0618 13:59:32.522912 (+ 82us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.522934 (+ 22us) write_op.cc:454] Released schema lock
0618 13:59:32.522942 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":25444224,"child_traces":[["op",{"apply.queue_time_us":67,"apply.run_cpu_time_us":514,"apply.run_wall_time_us":513,"num_ops":2,"prepare.queue_time_us":7972,"raft.queue_time_us":1623,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":283,"replication_time_us":34708,"spinlock_wait_cycles":28317952}]]}
I20260618 13:59:32.529178 29807 tablet.cc:1724] T 6b8f273def3e4b7caa4d624cf87a106c P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260618 13:59:32.532832 29456 raft_consensus.cc:493] T 118259402d7d48dda381cdd18eec0f1b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.532961 29456 raft_consensus.cc:515] T 118259402d7d48dda381cdd18eec0f1b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.533087 29808 tablet.cc:1724] T 9b75ece59d924e3db49e002bf0faa89c P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:32.533586 29456 leader_election.cc:290] T 118259402d7d48dda381cdd18eec0f1b P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:32.538187 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118259402d7d48dda381cdd18eec0f1b" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:32.539147 29644 raft_consensus.cc:515] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.540409 29644 leader_election.cc:290] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.541611 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d014b46a0f448b929cbc9287ead0a7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:32.541779 28410 raft_consensus.cc:3060] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.540550 29476 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 13:59:32.543953 28799 tablet_bootstrap.cc:492] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:32.544134 28799 ts_tablet_manager.cc:1403] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.358s user 0.077s sys 0.197s
I20260618 13:59:32.544852 28799 raft_consensus.cc:359] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.545039 28799 raft_consensus.cc:385] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.545099 28799 raft_consensus.cc:740] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.545737 28799 consensus_queue.cc:260] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.546219 28799 ts_tablet_manager.cc:1434] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:32.546643 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52006500f117417b8a2c979d4aeb077c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.547339 28662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5836982752b4209b72a372c1dfeb71c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.547713 29782 tablet.cc:1724] T 9b75ece59d924e3db49e002bf0faa89c P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:32.547873 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9ceb3af08fa4618b81cf6f89d7bd0df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.549119 29749 raft_consensus.cc:493] T c0a42aa779ae452e9420d730bf8d7d55 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.549273 29749 raft_consensus.cc:515] T c0a42aa779ae452e9420d730bf8d7d55 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:32.549866 29749 leader_election.cc:290] T c0a42aa779ae452e9420d730bf8d7d55 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:32.550086 29679 raft_consensus.cc:493] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.550212 29679 raft_consensus.cc:515] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.550782 29679 leader_election.cc:290] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.551460 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4c6537144d425f81fd1fafcd034b77" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:32.551930 28346 leader_election.cc:343] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.552683 28799 tablet_bootstrap.cc:492] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:32.553193 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a42aa779ae452e9420d730bf8d7d55" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:32.553452 28402 raft_consensus.cc:2393] T c0a42aa779ae452e9420d730bf8d7d55 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:32.553913 29679 raft_consensus.cc:493] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.554104 29679 raft_consensus.cc:515] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.554754 29679 leader_election.cc:290] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.555171 28799 tablet_bootstrap.cc:654] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.555431 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab100ebbe61a4321b9e76c730e5d46ed" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:32.555971 28346 leader_election.cc:343] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.556672 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab100ebbe61a4321b9e76c730e5d46ed" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:32.556794 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4c6537144d425f81fd1fafcd034b77" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:32.557279 28346 leader_election.cc:343] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.557403 28346 leader_election.cc:304] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:32.557765 28346 leader_election.cc:343] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.557881 28346 leader_election.cc:304] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.558190 29522 raft_consensus.cc:2749] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:32.558297 29679 raft_consensus.cc:2749] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:32.559056 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet d3dbf13544ce408ab97834aec5862390 with delete_type TABLET_DATA_DELETED (replaced by tablet ba18e879631947f88bd266681746ec82) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:32.560545 28656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43578: Illegal state: replica d6a169b16ced46ca9cfbc631709bb257 is not leader of this config: current role FOLLOWER
I20260618 13:59:32.561556 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet d3dbf13544ce408ab97834aec5862390 with delete_type TABLET_DATA_DELETED (replaced by tablet ba18e879631947f88bd266681746ec82) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:32.561837 29746 tablet_replica.cc:333] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:32.563733 28799 tablet_bootstrap.cc:492] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:32.563870 28799 ts_tablet_manager.cc:1403] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.003s
I20260618 13:59:32.563376 29749 raft_consensus.cc:493] T 512a9e63ba324a30b8d8c37e585ebdf8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.564536 29749 raft_consensus.cc:515] T 512a9e63ba324a30b8d8c37e585ebdf8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.564579 28799 raft_consensus.cc:359] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:32.564755 28799 raft_consensus.cc:385] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.564836 28799 raft_consensus.cc:740] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.565047 28799 consensus_queue.cc:260] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:32.565522 28799 ts_tablet_manager.cc:1434] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:32.565642 29749 leader_election.cc:290] T 512a9e63ba324a30b8d8c37e585ebdf8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:32.565829 28799 tablet_bootstrap.cc:492] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:32.566776 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512a9e63ba324a30b8d8c37e585ebdf8" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:32.570505 28799 tablet_bootstrap.cc:654] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.574259 28403 raft_consensus.cc:2468] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:32.575304 28617 leader_election.cc:304] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:32.575309 28797 tablet_bootstrap.cc:492] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:32.575959 28797 ts_tablet_manager.cc:1403] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.206s user 0.001s sys 0.002s
I20260618 13:59:32.558462 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:32.576941 28797 raft_consensus.cc:359] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.577195 28797 raft_consensus.cc:385] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.577286 28797 raft_consensus.cc:740] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.577070 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet ece7909d056c4ba8a5dca6423f0c1781 (table default.Table_26e7d8c4f91144268a2a43e83992667d [id=803d5220ce1a4dcfaf27d93c574be2ac]). Expected version 1 got 0
I20260618 13:59:32.577574 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet a1a06078974e4bfbbc4b949802a39bf1 (table default.Table_f958c4ce58c34bfda98454664e202af2 [id=7b747ef3b2494ca78c499c5bb75648d4]). Expected version 2 got 0
I20260618 13:59:32.577564 28797 consensus_queue.cc:260] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.577867 29607 raft_consensus.cc:2804] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:32.577905 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:32.578054 29607 raft_consensus.cc:697] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:32.578203 28797 ts_tablet_manager.cc:1434] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:32.578326 29607 consensus_queue.cc:237] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.578711 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:32.578927 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 15180aefba0544329daee74308ef5010 (table default.Table_97b6bb39b3bf4612be391b9dd78d1646 [id=42a2a24590834abc9537669aaf223a7c]). Expected version 1 got 0
I20260618 13:59:32.579190 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 98b42d23832545ae81a7cdff19e4689f (table default.Table_c99584ecc88948a097b39c22d68b3938 [id=7c39b3c432e944bca05f7b2070a79ff9]). Expected version 1 got 0
I20260618 13:59:32.579746 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a]). Expected version 1 got 0
I20260618 13:59:32.580224 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:32.580468 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9b75ece59d924e3db49e002bf0faa89c (table default.Table_031e1a03030b45c99c1270d612008f91 [id=387d3ff1762749efad926181fe4c746a]). Expected version 1 got 0
I20260618 13:59:32.580713 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 6b8f273def3e4b7caa4d624cf87a106c (table default.Table_5159a242b12044fc9a0b3e535ea40050 [id=cfb4ca8c4e884ccc8602be52d0b39100]). Expected version 4 got 0
I20260618 13:59:32.581355 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 512a9e63ba324a30b8d8c37e585ebdf8 (table default.Table_c70bc1c98b3b49d7aa3080904667732d [id=901874cb80e74ba589679bb7b95c8fe4]). Expected version 1 got 0
I20260618 13:59:32.582304 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:32.582899 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:32.583200 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:32.583864 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet dc36fbea853a4d4e9f61909200239aed (table default.Table_dbc6162c06f74e71b792bd1f7b2388fe [id=49576dabdecf41aca1b15ff3fc0c113c]). Expected version 6 got 0
I20260618 13:59:32.584281 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 39b06ae7444a48dbbec9e096fd7e4052 (table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15]). Expected version 2 got 0
I20260618 13:59:32.584524 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:32.584787 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet cea4bc13eabc4dd98b4f96723498014c (table default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c]). Expected version 1 got 0
I20260618 13:59:32.585099 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:32.585536 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:32.576588 29020 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_74fc7c21dc664ff8bfe0d25aee399bdb" } modify_external_catalogs: true
I20260618 13:59:32.585852 29020 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_74fc7c21dc664ff8bfe0d25aee399bdb" } modify_external_catalogs: true
I20260618 13:59:32.586607 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:32.588899 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 1f81428f804a485c967026c4527fd250 (table default.Table_706eec6806a94d3f8d5b60708a0c1cda [id=b817db93d50144d7b7c238550eb53b10]). Expected version 1 got 0
I20260618 13:59:32.590009 29476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 13:59:32.592979 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:32.593521 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:32.594201 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:32.596194 29759 raft_consensus.cc:493] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:32.596432 29759 raft_consensus.cc:515] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.597141 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:32.597872 28797 tablet_bootstrap.cc:492] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:32.597879 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:32.598362 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:32.599109 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_efb2b8e451084b7ba0de07e0a6e538c2 [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 1 got 0
I20260618 13:59:32.599877 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:32.600211 28797 tablet_bootstrap.cc:654] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.605338 28798 tablet_bootstrap.cc:492] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:32.605541 28798 ts_tablet_manager.cc:1403] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.087s user 0.008s sys 0.007s
I20260618 13:59:32.606467 28798 raft_consensus.cc:359] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.606771 28798 raft_consensus.cc:385] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.606933 28798 raft_consensus.cc:740] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.607206 28798 consensus_queue.cc:260] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.607985 28798 ts_tablet_manager.cc:1434] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 13:59:32.608316 28798 tablet_bootstrap.cc:492] T 15180aefba0544329daee74308ef5010 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:32.608810 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:32.608960 28797 tablet_bootstrap.cc:492] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:32.609151 28797 ts_tablet_manager.cc:1403] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260618 13:59:32.606559 29759 leader_election.cc:290] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:32.610281 28797 raft_consensus.cc:359] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.610528 28797 raft_consensus.cc:385] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.610633 28797 raft_consensus.cc:740] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.610881 28797 consensus_queue.cc:260] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.612188 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6dd422c05324c989e29b20ae2afabad" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:32.612751 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:32.613468 29026 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:32.614264 29811 tablet.cc:1724] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:32.614806 28798 tablet_bootstrap.cc:654] T 15180aefba0544329daee74308ef5010 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.619359 28799 tablet_bootstrap.cc:492] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:32.619527 28799 ts_tablet_manager.cc:1403] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.054s user 0.011s sys 0.024s
I20260618 13:59:32.620213 28799 raft_consensus.cc:359] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.620393 28799 raft_consensus.cc:385] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.620455 28799 raft_consensus.cc:740] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.620662 28799 consensus_queue.cc:260] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.621129 28799 ts_tablet_manager.cc:1434] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:32.624569 28799 tablet_bootstrap.cc:492] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:32.624836 28797 ts_tablet_manager.cc:1434] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.016s user 0.003s sys 0.000s
I20260618 13:59:32.625211 28797 tablet_bootstrap.cc:492] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:32.628995 28373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5836982752b4209b72a372c1dfeb71c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.630225 28797 tablet_bootstrap.cc:654] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.634181 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9ceb3af08fa4618b81cf6f89d7bd0df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.634884 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52006500f117417b8a2c979d4aeb077c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.639326 29755 raft_consensus.cc:2749] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:32.642810 28410 raft_consensus.cc:2468] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:32.647157 28799 tablet_bootstrap.cc:654] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.651131 29749 raft_consensus.cc:493] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.651361 29749 raft_consensus.cc:515] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.652701 29749 leader_election.cc:290] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:32.655282 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9fa9675234db2abff694a517c0ea8" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:32.655571 28410 raft_consensus.cc:2468] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:32.656170 28483 leader_election.cc:304] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2; no voters:
I20260618 13:59:32.656536 29759 raft_consensus.cc:2804] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:32.656642 29759 raft_consensus.cc:493] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:32.656759 29759 raft_consensus.cc:3060] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.678699 29749 raft_consensus.cc:493] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.678900 29749 raft_consensus.cc:515] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.679594 29749 leader_election.cc:290] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:32.680631 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a42aa779ae452e9420d730bf8d7d55" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:32.680728 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512a9e63ba324a30b8d8c37e585ebdf8" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:32.680727 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "118259402d7d48dda381cdd18eec0f1b" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:32.681273 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6dd422c05324c989e29b20ae2afabad" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:32.681963 28681 raft_consensus.cc:1275] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257 [term 2 FOLLOWER]: Refusing update from remote peer 97859748af2c41879ba2a668dda999d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260618 13:59:32.682693 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9fa9675234db2abff694a517c0ea8" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:32.682878 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f3d2f236fd44ab8af0524ea282cfa94" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:32.683655 29719 consensus_queue.cc:1048] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:32.687259 28799 tablet_bootstrap.cc:492] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:32.687548 28799 ts_tablet_manager.cc:1403] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.063s user 0.005s sys 0.005s
I20260618 13:59:32.688611 28799 raft_consensus.cc:359] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.688915 28799 raft_consensus.cc:385] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.689726 28799 raft_consensus.cc:740] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.690042 28799 consensus_queue.cc:260] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.693749 29749 raft_consensus.cc:493] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.693951 29749 raft_consensus.cc:515] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.694851 28799 ts_tablet_manager.cc:1434] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260618 13:59:32.695092 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827cc0987bd541759e334fb620f04ed5" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:32.695170 28799 tablet_bootstrap.cc:492] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:32.696245 29749 leader_election.cc:290] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:32.696938 28799 tablet_bootstrap.cc:654] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.700789 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d014b46a0f448b929cbc9287ead0a7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:32.702015 28549 raft_consensus.cc:3060] T 118259402d7d48dda381cdd18eec0f1b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.702720 28544 raft_consensus.cc:3060] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.703972 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 51b91f053bcc42bb829bb8180e232c8e (DEFAULT_TABLE table=default.Table_49b3aa53eeeb4a208dcec66719481e47 [id=cf401c4b51d84ba48f431fbdbc6eb571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.705233 28513 tablet_service.cc:1511] Processing CreateTablet for tablet b2a6fabad00642528913e735929e295e (DEFAULT_TABLE table=default.Table_72b47dac4bd24e24a0eae1a3de512e9f [id=1563a68dc96e4110843b002d403dfcf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.706981 28372 tablet_service.cc:1511] Processing CreateTablet for tablet 51b91f053bcc42bb829bb8180e232c8e (DEFAULT_TABLE table=default.Table_49b3aa53eeeb4a208dcec66719481e47 [id=cf401c4b51d84ba48f431fbdbc6eb571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.708909 28377 tablet_service.cc:1511] Processing CreateTablet for tablet b2a6fabad00642528913e735929e295e (DEFAULT_TABLE table=default.Table_72b47dac4bd24e24a0eae1a3de512e9f [id=1563a68dc96e4110843b002d403dfcf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.710012 28510 tablet_service.cc:1511] Processing CreateTablet for tablet 0e6f3f8431cd49b5bf1306bc7609761d (DEFAULT_TABLE table=default.Table_909b56b640ef4b5db58e13916f138fb0 [id=26ac245166234803b7ed1acfdbb8df65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.712527 29679 raft_consensus.cc:493] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.712694 29679 raft_consensus.cc:515] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.712885 28386 tablet_service.cc:1511] Processing CreateTablet for tablet 0e6f3f8431cd49b5bf1306bc7609761d (DEFAULT_TABLE table=default.Table_909b56b640ef4b5db58e13916f138fb0 [id=26ac245166234803b7ed1acfdbb8df65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.713424 29679 leader_election.cc:290] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.713673 28410 raft_consensus.cc:1275] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b [term 2 FOLLOWER]: Refusing update from remote peer 97859748af2c41879ba2a668dda999d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260618 13:59:32.714259 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56650c9e20e24a28821cca4279bfde58" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:32.715231 29755 consensus_queue.cc:1048] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:32.715662 28514 tablet_service.cc:1511] Processing CreateTablet for tablet d3ed5c1137434fcc80ed0ca56a323be7 (DEFAULT_TABLE table=default.Table_4d777a002dff4c77b022c9d8f05a3804 [id=c8e77e70f3aa451587701026197b5b5f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:32.717731 28346 leader_election.cc:343] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.719791 28381 tablet_service.cc:1511] Processing CreateTablet for tablet d3ed5c1137434fcc80ed0ca56a323be7 (DEFAULT_TABLE table=default.Table_4d777a002dff4c77b022c9d8f05a3804 [id=c8e77e70f3aa451587701026197b5b5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.720187 28522 tablet_service.cc:1511] Processing CreateTablet for tablet 72c04f55890648f0a8184f3783243f7c (DEFAULT_TABLE table=default.Table_8783a8d8c526428281436b62b2581a33 [id=0d83fc68255c458587390c58b12e00d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.722101 28388 tablet_service.cc:1511] Processing CreateTablet for tablet 72c04f55890648f0a8184f3783243f7c (DEFAULT_TABLE table=default.Table_8783a8d8c526428281436b62b2581a33 [id=0d83fc68255c458587390c58b12e00d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.722784 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51b91f053bcc42bb829bb8180e232c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.726868 28513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2a6fabad00642528913e735929e295e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.727123 29679 raft_consensus.cc:493] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.727288 29679 raft_consensus.cc:515] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.727402 28510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e6f3f8431cd49b5bf1306bc7609761d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.727818 28514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3ed5c1137434fcc80ed0ca56a323be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.727820 29679 leader_election.cc:290] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.728242 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72c04f55890648f0a8184f3783243f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.728700 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d8b3cfac0f648cd8b1c51102132c676" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:32.729566 28346 leader_election.cc:343] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.733655 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56650c9e20e24a28821cca4279bfde58" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:32.734084 28346 leader_election.cc:343] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.734220 28346 leader_election.cc:304] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.734599 28545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d8b3cfac0f648cd8b1c51102132c676" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:32.734843 29679 raft_consensus.cc:2749] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:32.735232 28346 leader_election.cc:343] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.735355 28346 leader_election.cc:304] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.735670 29679 raft_consensus.cc:2749] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:32.738653 28617 leader_election.cc:304] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
W20260618 13:59:32.741748 28617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 98330.2us
I20260618 13:59:32.741973 29303 raft_consensus.cc:2804] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:32.743340 28654 tablet_service.cc:1511] Processing CreateTablet for tablet b2a6fabad00642528913e735929e295e (DEFAULT_TABLE table=default.Table_72b47dac4bd24e24a0eae1a3de512e9f [id=1563a68dc96e4110843b002d403dfcf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.742105 29303 raft_consensus.cc:697] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:32.747018 28663 tablet_service.cc:1511] Processing CreateTablet for tablet 0e6f3f8431cd49b5bf1306bc7609761d (DEFAULT_TABLE table=default.Table_909b56b640ef4b5db58e13916f138fb0 [id=26ac245166234803b7ed1acfdbb8df65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.747251 29303 consensus_queue.cc:237] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.746994 29640 raft_consensus.cc:493] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.747535 29640 raft_consensus.cc:515] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.749459 29640 leader_election.cc:290] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.750918 28406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6de1293572f4e75af3417f281b9a142" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:32.751122 28406 raft_consensus.cc:2468] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:32.751088 28655 tablet_service.cc:1511] Processing CreateTablet for tablet d3ed5c1137434fcc80ed0ca56a323be7 (DEFAULT_TABLE table=default.Table_4d777a002dff4c77b022c9d8f05a3804 [id=c8e77e70f3aa451587701026197b5b5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.751506 28649 tablet_service.cc:1511] Processing CreateTablet for tablet 72c04f55890648f0a8184f3783243f7c (DEFAULT_TABLE table=default.Table_8783a8d8c526428281436b62b2581a33 [id=0d83fc68255c458587390c58b12e00d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.751576 28617 leader_election.cc:304] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:32.750483 29714 consensus_queue.cc:1048] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:32.751924 29719 raft_consensus.cc:2804] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:32.752003 29719 raft_consensus.cc:493] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:32.752077 29719 raft_consensus.cc:3060] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.752004 28656 tablet_service.cc:1511] Processing CreateTablet for tablet 51b91f053bcc42bb829bb8180e232c8e (DEFAULT_TABLE table=default.Table_49b3aa53eeeb4a208dcec66719481e47 [id=cf401c4b51d84ba48f431fbdbc6eb571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.752389 29745 ts_tablet_manager.cc:1929] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:32.752493 29745 log.cc:1199] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/f278d0acbb0c4a0ca61640bc142e6335
I20260618 13:59:32.752823 29745 ts_tablet_manager.cc:1950] T f278d0acbb0c4a0ca61640bc142e6335 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:32.762280 29749 raft_consensus.cc:493] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.762863 29749 raft_consensus.cc:515] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.762609 28512 tablet_service.cc:1511] Processing CreateTablet for tablet 7c50ab60a26647558da509236aac1e8b (DEFAULT_TABLE table=default.Table_a136205448ca4d778d01b1456898276f [id=b9cd89be31284caa8240881ddf1736da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.764206 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:32.765138 29749 leader_election.cc:290] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:32.765875 28483 leader_election.cc:343] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: fca76774f119435899188313a33dad98
I20260618 13:59:32.767863 28385 tablet_service.cc:1511] Processing CreateTablet for tablet 7c50ab60a26647558da509236aac1e8b (DEFAULT_TABLE table=default.Table_a136205448ca4d778d01b1456898276f [id=b9cd89be31284caa8240881ddf1736da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.769428 28524 tablet_service.cc:1511] Processing CreateTablet for tablet b44c3b0c152442a78edc4736da2cb15c (DEFAULT_TABLE table=default.Table_1674937bcc0342af83719008ac10377d [id=6ec6ddee6fee41dd997b1fae63d89f9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.771555 29640 consensus_queue.cc:1048] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:32.772632 28652 tablet_service.cc:1511] Processing CreateTablet for tablet 7c50ab60a26647558da509236aac1e8b (DEFAULT_TABLE table=default.Table_a136205448ca4d778d01b1456898276f [id=b9cd89be31284caa8240881ddf1736da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.773154 28659 tablet_service.cc:1511] Processing CreateTablet for tablet b44c3b0c152442a78edc4736da2cb15c (DEFAULT_TABLE table=default.Table_1674937bcc0342af83719008ac10377d [id=6ec6ddee6fee41dd997b1fae63d89f9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.773311 28384 tablet_service.cc:1511] Processing CreateTablet for tablet b44c3b0c152442a78edc4736da2cb15c (DEFAULT_TABLE table=default.Table_1674937bcc0342af83719008ac10377d [id=6ec6ddee6fee41dd997b1fae63d89f9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:32.769860 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:32.777220 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:32.778164 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:32.778786 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:32.779515 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_efb2b8e451084b7ba0de07e0a6e538c2 [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 1 got 0
I20260618 13:59:32.780643 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 98b42d23832545ae81a7cdff19e4689f (table default.Table_c99584ecc88948a097b39c22d68b3938 [id=7c39b3c432e944bca05f7b2070a79ff9]). Expected version 1 got 0
I20260618 13:59:32.782207 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
W20260618 13:59:32.782267 28618 leader_election.cc:343] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:32.782974 28618 consensus_peers.cc:597] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:32.783484 29018 catalog_manager.cc:5696] T 118259402d7d48dda381cdd18eec0f1b P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
W20260618 13:59:32.783787 28484 leader_election.cc:343] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: c84189a4b1194c87bbdf81f6e0ed56e4
I20260618 13:59:32.783903 28484 leader_election.cc:304] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.784196 29749 raft_consensus.cc:2749] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:32.784777 28484 leader_election.cc:304] T c0a42aa779ae452e9420d730bf8d7d55 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.784984 29749 raft_consensus.cc:3060] T c0a42aa779ae452e9420d730bf8d7d55 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.785548 28484 leader_election.cc:304] T 512a9e63ba324a30b8d8c37e585ebdf8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.785648 28799 tablet_bootstrap.cc:492] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:32.785807 29755 raft_consensus.cc:2749] T 512a9e63ba324a30b8d8c37e585ebdf8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:32.786142 28484 leader_election.cc:304] T 118259402d7d48dda381cdd18eec0f1b P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.786597 28484 leader_election.cc:304] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.786805 29754 raft_consensus.cc:2749] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:32.787040 29754 consensus_queue.cc:1048] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260618 13:59:32.785543 29018 catalog_manager.cc:5696] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:32.787569 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
W20260618 13:59:32.788625 28484 leader_election.cc:343] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.789011 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 15180aefba0544329daee74308ef5010 (table default.Table_97b6bb39b3bf4612be391b9dd78d1646 [id=42a2a24590834abc9537669aaf223a7c]). Expected version 1 got 0
I20260618 13:59:32.790000 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:32.791257 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 9b75ece59d924e3db49e002bf0faa89c (table default.Table_031e1a03030b45c99c1270d612008f91 [id=387d3ff1762749efad926181fe4c746a]). Expected version 1 got 0
I20260618 13:59:32.792316 28797 tablet_bootstrap.cc:492] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:32.792450 28797 ts_tablet_manager.cc:1403] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.167s user 0.031s sys 0.037s
I20260618 13:59:32.793670 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:32.794162 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:32.795069 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 512a9e63ba324a30b8d8c37e585ebdf8 (table default.Table_c70bc1c98b3b49d7aa3080904667732d [id=901874cb80e74ba589679bb7b95c8fe4]). Expected version 1 got 0
I20260618 13:59:32.796321 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:32.798221 28797 raft_consensus.cc:359] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.798395 28797 raft_consensus.cc:385] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.798471 28797 raft_consensus.cc:740] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.798632 28797 consensus_queue.cc:260] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.799651 28797 ts_tablet_manager.cc:1434] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260618 13:59:32.800318 28797 tablet_bootstrap.cc:492] T 8fd9fa9675234db2abff694a517c0ea8 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:32.800962 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:32.801618 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:32.801728 28797 tablet_bootstrap.cc:654] T 8fd9fa9675234db2abff694a517c0ea8 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.810894 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f3d2f236fd44ab8af0524ea282cfa94" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:32.812222 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "827cc0987bd541759e334fb620f04ed5" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:32.822994 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6de1293572f4e75af3417f281b9a142" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:32.823829 28537 raft_consensus.cc:3060] T 512a9e63ba324a30b8d8c37e585ebdf8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.824934 28484 leader_election.cc:304] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:32.825947 28618 consensus_peers.cc:597] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:32.826367 28618 leader_election.cc:343] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.830981 29719 raft_consensus.cc:515] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.831591 29719 leader_election.cc:290] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.802656 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet dc36fbea853a4d4e9f61909200239aed (table default.Table_dbc6162c06f74e71b792bd1f7b2388fe [id=49576dabdecf41aca1b15ff3fc0c113c]). Expected version 6 got 0
I20260618 13:59:32.836194 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:32.836946 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:32.837934 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet a1a06078974e4bfbbc4b949802a39bf1 (table default.Table_f958c4ce58c34bfda98454664e202af2 [id=7b747ef3b2494ca78c499c5bb75648d4]). Expected version 2 got 0
I20260618 13:59:32.833803 28798 tablet_bootstrap.cc:492] T 15180aefba0544329daee74308ef5010 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:32.838995 28798 ts_tablet_manager.cc:1403] T 15180aefba0544329daee74308ef5010 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.231s user 0.014s sys 0.029s
I20260618 13:59:32.839141 28484 leader_election.cc:304] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.839875 28798 raft_consensus.cc:359] T 15180aefba0544329daee74308ef5010 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.840101 28798 raft_consensus.cc:385] T 15180aefba0544329daee74308ef5010 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.840193 28798 raft_consensus.cc:740] T 15180aefba0544329daee74308ef5010 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.840504 28798 consensus_queue.cc:260] T 15180aefba0544329daee74308ef5010 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.843992 29018 catalog_manager.cc:5696] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:32.844893 29839 raft_consensus.cc:2749] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:32.844924 28797 tablet_bootstrap.cc:492] T 8fd9fa9675234db2abff694a517c0ea8 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:32.845036 28797 ts_tablet_manager.cc:1403] T 8fd9fa9675234db2abff694a517c0ea8 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.045s user 0.001s sys 0.004s
I20260618 13:59:32.845284 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6de1293572f4e75af3417f281b9a142" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:32.845523 28410 raft_consensus.cc:3060] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.845671 28797 raft_consensus.cc:359] T 8fd9fa9675234db2abff694a517c0ea8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.845870 28797 raft_consensus.cc:385] T 8fd9fa9675234db2abff694a517c0ea8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.845934 28797 raft_consensus.cc:740] T 8fd9fa9675234db2abff694a517c0ea8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.847994 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet ece7909d056c4ba8a5dca6423f0c1781 (table default.Table_26e7d8c4f91144268a2a43e83992667d [id=803d5220ce1a4dcfaf27d93c574be2ac]). Expected version 1 got 0
W20260618 13:59:32.848829 28484 leader_election.cc:343] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: fca76774f119435899188313a33dad98
I20260618 13:59:32.850107 28372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51b91f053bcc42bb829bb8180e232c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.850351 28377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2a6fabad00642528913e735929e295e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.850446 28484 leader_election.cc:304] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:32.850646 29839 raft_consensus.cc:2749] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:32.850860 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e6f3f8431cd49b5bf1306bc7609761d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.851230 28381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3ed5c1137434fcc80ed0ca56a323be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.851403 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72c04f55890648f0a8184f3783243f7c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:32.851507 28618 consensus_peers.cc:597] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:32.852047 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:32.852283 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 6b8f273def3e4b7caa4d624cf87a106c (table default.Table_5159a242b12044fc9a0b3e535ea40050 [id=cfb4ca8c4e884ccc8602be52d0b39100]). Expected version 4 got 0
I20260618 13:59:32.852439 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 39b06ae7444a48dbbec9e096fd7e4052 (table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15]). Expected version 2 got 0
I20260618 13:59:32.852685 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a]). Expected version 1 got 0
I20260618 13:59:32.852816 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:32.852939 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet cea4bc13eabc4dd98b4f96723498014c (table default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c]). Expected version 1 got 0
I20260618 13:59:32.853083 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:32.853315 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:32.846132 28797 consensus_queue.cc:260] T 8fd9fa9675234db2abff694a517c0ea8 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.856845 28797 ts_tablet_manager.cc:1434] T 8fd9fa9675234db2abff694a517c0ea8 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260618 13:59:32.857110 28797 tablet_bootstrap.cc:492] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:32.860862 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 1f81428f804a485c967026c4527fd250 (table default.Table_706eec6806a94d3f8d5b60708a0c1cda [id=b817db93d50144d7b7c238550eb53b10]). Expected version 1 got 0
I20260618 13:59:32.861055 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:32.861317 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2a6fabad00642528913e735929e295e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.861873 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e6f3f8431cd49b5bf1306bc7609761d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.862317 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3ed5c1137434fcc80ed0ca56a323be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.862800 28649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72c04f55890648f0a8184f3783243f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.863234 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51b91f053bcc42bb829bb8180e232c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.863627 28652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c50ab60a26647558da509236aac1e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.863999 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b44c3b0c152442a78edc4736da2cb15c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.864270 28797 tablet_bootstrap.cc:654] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.865499 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:32.867451 29019 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet e3d8616846c649b6a969b971db0d03db
I20260618 13:59:32.867940 28512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c50ab60a26647558da509236aac1e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.868039 29019 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 50) took 2268 ms. Trace:
I20260618 13:59:32.868479 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b44c3b0c152442a78edc4736da2cb15c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.868140 29019 rpcz_store.cc:276] 0618 13:59:30.599863 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:31.971441 (+1371578us) service_pool.cc:224] Handling call
0618 13:59:32.084023 (+112582us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb46856b6 00007fcbab0e2486 00007fcbab0e21b5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:32.084485 (+ 462us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:32.084641 (+ 156us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:32.084712 (+ 71us) catalog_manager.cc:2849] Locking tablets
0618 13:59:32.084896 (+ 184us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:32.329352 (+244456us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:32.393820 (+ 64468us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:32.867384 (+473564us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:32.868001 (+ 617us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:32.277716 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:32.277907 (+ 191us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:32.277914 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:32.277918 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:32.278064 (+ 146us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:32.278080 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:32.278086 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:32.278232 (+ 146us) tablet.cc:701] Row locks acquired
0618 13:59:32.278235 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:32.278281 (+ 46us) write_op.cc:270] Start()
0618 13:59:32.278325 (+ 44us) write_op.cc:276] Timestamp: P: 1781791172278247 usec, L: 0
0618 13:59:32.278329 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:32.278584 (+ 255us) log.cc:844] Serialized 728 byte log entry
0618 13:59:32.323753 (+ 45169us) op_driver.cc:464] REPLICATION: finished
0618 13:59:32.327749 (+ 3996us) write_op.cc:301] APPLY: starting
0618 13:59:32.327770 (+ 21us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:32.327818 (+ 48us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:32.327850 (+ 32us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:32.327925 (+ 75us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:32.327958 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:32.327965 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:32.328065 (+ 100us) log.cc:844] Serialized 29 byte log entry
0618 13:59:32.328139 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.328157 (+ 18us) write_op.cc:454] Released schema lock
0618 13:59:32.328164 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:32.328253 (+ 89us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.328257 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":258211584,"child_traces":[["op",{"apply.queue_time_us":3927,"apply.run_cpu_time_us":499,"apply.run_wall_time_us":499,"num_ops":2,"prepare.queue_time_us":192070,"prepare.run_cpu_time_us":1001,"prepare.run_wall_time_us":2473,"raft.queue_time_us":477,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"replication_time_us":45377}]]}
I20260618 13:59:32.868966 29019 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_9cf9f72ae27b44bc9aa43303a7103214" } new_table_name: "default.Table_25a9b51a35344e3ca6689e9847de130c" modify_external_catalogs: true
W20260618 13:59:32.874243 29011 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f45556a59c424ca2bc30956566374c89 consensus state has no leader
I20260618 13:59:32.874471 29011 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 60) took 1023 ms. Trace:
I20260618 13:59:32.874660 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet f278d0acbb0c4a0ca61640bc142e6335 (table default.Table_8783a8d8c526428281436b62b2581a33 [id=0d83fc68255c458587390c58b12e00d7]) successfully deleted
I20260618 13:59:32.874576 29011 rpcz_store.cc:276] 0618 13:59:31.851327 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.486792 (+635465us) service_pool.cc:224] Handling call
0618 13:59:32.488533 (+ 1741us) spinlock_profiling.cc:243] Waited 1.32 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb4686956 00007fcbab0e4146 00007fcbab0e3e75 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:32.488837 (+ 304us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:32.489103 (+ 266us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:32.489276 (+ 173us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:32.489366 (+ 90us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:32.620854 (+131488us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:32.874393 (+253539us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:32.535032 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:32.535178 (+ 146us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:32.535184 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:32.535188 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:32.535294 (+ 106us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:32.535310 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:32.535317 (+ 7us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:32.535403 (+ 86us) tablet.cc:701] Row locks acquired
0618 13:59:32.535407 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:32.535445 (+ 38us) write_op.cc:270] Start()
0618 13:59:32.535484 (+ 39us) write_op.cc:276] Timestamp: P: 1781791172535386 usec, L: 0
0618 13:59:32.535490 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:32.569349 (+ 33859us) log.cc:844] Serialized 405 byte log entry
0618 13:59:32.570185 (+ 836us) spinlock_profiling.cc:243] Waited 33.2 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fcbac75ef22 00007fcbadb40305 00007fcbadb3ee29 00007fcbadb47250 00007fcbadb4705d 0000000000aa681c 00007fcba07299ff 00007fcba072cd1b 00007fcba072cb98 0000000000aa681c
0618 13:59:32.620288 (+ 50103us) op_driver.cc:464] REPLICATION: finished
0618 13:59:32.620368 (+ 80us) write_op.cc:301] APPLY: starting
0618 13:59:32.620386 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:32.620429 (+ 43us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:32.620431 (+ 2us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:32.620482 (+ 51us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:32.620513 (+ 31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:32.620520 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:32.620601 (+ 81us) log.cc:844] Serialized 23 byte log entry
0618 13:59:32.620661 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.620684 (+ 23us) write_op.cc:454] Released schema lock
0618 13:59:32.620708 (+ 24us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":3032448,"child_traces":[["op",{"apply.queue_time_us":33,"apply.run_cpu_time_us":425,"apply.run_wall_time_us":424,"num_ops":1,"prepare.queue_time_us":45192,"prepare.run_cpu_time_us":1625,"prepare.run_wall_time_us":35242,"raft.queue_time_us":86,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"replication_time_us":84763,"spinlock_wait_cycles":76310400}]]}
I20260618 13:59:32.875167 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6de1293572f4e75af3417f281b9a142" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:32.875514 29011 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_659a3d495ed44be4bfb2193816362e25" } modify_external_catalogs: true
I20260618 13:59:32.875680 29011 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_659a3d495ed44be4bfb2193816362e25" } modify_external_catalogs: true
W20260618 13:59:32.876878 29017 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3a783590159f4a80964fbbcb4a24eef9 consensus state has no leader
I20260618 13:59:32.877025 29017 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 43) took 2518 ms. Trace:
I20260618 13:59:32.877146 29017 rpcz_store.cc:276] 0618 13:59:30.358471 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:31.914400 (+1555929us) service_pool.cc:224] Handling call
0618 13:59:31.944208 (+ 29808us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:31.998798 (+ 54590us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:31.999015 (+ 217us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:31.999102 (+ 87us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:32.156952 (+157850us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:32.876988 (+720036us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:31.999693 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:31.999793 (+ 100us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:31.999801 (+ 8us) write_op.cc:435] Acquired schema lock
0618 13:59:31.999804 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:31.999934 (+ 130us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:31.999952 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:31.999960 (+ 8us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:32.000069 (+ 109us) tablet.cc:701] Row locks acquired
0618 13:59:32.000073 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:32.000117 (+ 44us) write_op.cc:270] Start()
0618 13:59:32.000160 (+ 43us) write_op.cc:276] Timestamp: P: 1781791172000059 usec, L: 0
0618 13:59:32.000164 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:32.079486 (+ 79322us) log.cc:844] Serialized 405 byte log entry
0618 13:59:32.079804 (+ 318us) spinlock_profiling.cc:243] Waited 79.1 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fcbac75ef22 00007fcbadb40305 00007fcbadb3ee29 00007fcbadb47250 00007fcbadb4705d 0000000000aa681c 00007fcba07299ff 00007fcba072cd1b 00007fcba072cb98 0000000000aa681c
0618 13:59:32.151875 (+ 72071us) op_driver.cc:464] REPLICATION: finished
0618 13:59:32.152115 (+ 240us) write_op.cc:301] APPLY: starting
0618 13:59:32.152134 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:32.152177 (+ 43us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:32.152180 (+ 3us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:32.152235 (+ 55us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:32.152268 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:32.152275 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:32.152359 (+ 84us) log.cc:844] Serialized 23 byte log entry
0618 13:59:32.152422 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.152442 (+ 20us) write_op.cc:454] Released schema lock
0618 13:59:32.152449 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:32.152520 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.152524 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":187,"apply.run_cpu_time_us":394,"apply.run_wall_time_us":393,"num_ops":1,"prepare.queue_time_us":58,"prepare.run_cpu_time_us":2219,"prepare.run_wall_time_us":80199,"raft.queue_time_us":3210,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":294,"replication_time_us":151676,"spinlock_wait_cycles":181855104}]]}
I20260618 13:59:32.877936 29017 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_ab57b782a92747e78e2bfe28480aa0ec" } modify_external_catalogs: true
I20260618 13:59:32.878085 29017 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_ab57b782a92747e78e2bfe28480aa0ec" } modify_external_catalogs: true
I20260618 13:59:32.888298 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c50ab60a26647558da509236aac1e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.888342 28799 ts_tablet_manager.cc:1403] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.193s user 0.015s sys 0.008s
I20260618 13:59:32.889091 28799 raft_consensus.cc:359] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:32.889267 28799 raft_consensus.cc:385] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.889321 28799 raft_consensus.cc:740] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.889508 28799 consensus_queue.cc:260] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:32.889950 28799 ts_tablet_manager.cc:1434] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 13:59:32.890208 28799 tablet_bootstrap.cc:492] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:32.892055 28618 leader_election.cc:343] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.893661 28799 tablet_bootstrap.cc:654] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.895040 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b44c3b0c152442a78edc4736da2cb15c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:32.896580 29303 raft_consensus.cc:493] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.896710 29303 raft_consensus.cc:515] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.897492 29303 leader_election.cc:290] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.898692 29007 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 55ae29c3b7ed44cfa659130745037aa6
I20260618 13:59:32.899137 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b51d0851efb34eaa9e9e4be6b292ef62" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:32.899202 29007 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 48) took 2394 ms. Trace:
I20260618 13:59:32.899312 29007 rpcz_store.cc:276] 0618 13:59:30.504856 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:31.939349 (+1434493us) service_pool.cc:224] Handling call
0618 13:59:31.939821 (+ 472us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:31.939967 (+ 146us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:31.940030 (+ 63us) catalog_manager.cc:2849] Locking tablets
0618 13:59:32.001105 (+ 61075us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:32.153213 (+152108us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:32.153302 (+ 89us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:32.898614 (+745312us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:32.899162 (+ 548us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:32.079948 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:32.080081 (+ 133us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:32.080088 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:32.080091 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:32.080219 (+ 128us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:32.080229 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:32.080234 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:32.080346 (+ 112us) tablet.cc:701] Row locks acquired
0618 13:59:32.080348 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:32.080390 (+ 42us) write_op.cc:270] Start()
0618 13:59:32.080425 (+ 35us) write_op.cc:276] Timestamp: P: 1781791172080326 usec, L: 0
0618 13:59:32.080428 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:32.080648 (+ 220us) log.cc:844] Serialized 728 byte log entry
0618 13:59:32.151954 (+ 71306us) op_driver.cc:464] REPLICATION: finished
0618 13:59:32.152593 (+ 639us) write_op.cc:301] APPLY: starting
0618 13:59:32.152604 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:32.152636 (+ 32us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:32.152684 (+ 48us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:32.152737 (+ 53us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:32.152764 (+ 27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:32.152771 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:32.152825 (+ 54us) log.cc:844] Serialized 29 byte log entry
0618 13:59:32.152863 (+ 38us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.152878 (+ 15us) write_op.cc:454] Released schema lock
0618 13:59:32.152883 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:32.204292 (+ 51409us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.204301 (+ 9us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":608,"apply.run_cpu_time_us":342,"apply.run_wall_time_us":341,"num_ops":2,"prepare.queue_time_us":78172,"prepare.run_cpu_time_us":9182,"prepare.run_wall_time_us":124335,"raft.queue_time_us":2219,"raft.run_cpu_time_us":8,"raft.run_wall_time_us":9,"replication_time_us":71509,"thread_start_us":123122,"threads_started":1}]]}
I20260618 13:59:32.900149 29007 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_b703123b497f445b89066c7ec657f91c" } new_table_name: "default.Table_fed3fd1cd60c4b50b1c30442807161fe" modify_external_catalogs: true
W20260618 13:59:32.900501 28618 leader_election.cc:343] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:32.900856 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet e3d8616846c649b6a969b971db0d03db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.900980 28406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b51d0851efb34eaa9e9e4be6b292ef62" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:32.901190 28406 raft_consensus.cc:2468] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:32.901675 28617 leader_election.cc:304] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:32.902609 29303 raft_consensus.cc:2804] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:32.902704 29303 raft_consensus.cc:493] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:32.902777 29303 raft_consensus.cc:3060] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.906311 28382 tablet_service.cc:1558] Processing DeleteTablet for tablet 316f470ea82f42fca48e4e96d49c9574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.906554 28382 tablet_service.cc:1558] Processing DeleteTablet for tablet fddea8fba4da40d7a5780e6750caf5e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.906755 28387 tablet_service.cc:1558] Processing DeleteTablet for tablet 897bf6d6bbee4ad89c296325d94a3e53 with delete_type TABLET_DATA_DELETED (replaced by tablet e196dd3c7fe14524a50152643ddbc2ad) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.906966 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet f89c9954d48e407c96c294c26644e25c with delete_type TABLET_DATA_DELETED (replaced by tablet b0d3d363766546c4a97b0763bad11f71) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.907162 28374 tablet_service.cc:1558] Processing DeleteTablet for tablet dd2385e6752d49429422fc854539e9c6 with delete_type TABLET_DATA_DELETED (replaced by tablet c38a2156a55e4e5eb91d101e9a61eecd) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.907351 28376 tablet_service.cc:1558] Processing DeleteTablet for tablet 283523e5b22843ba935eabe6283d3a99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.907532 28379 tablet_service.cc:1558] Processing DeleteTablet for tablet c7f42997416841a7b2c1ad347d2e9282 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.907701 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet d502d67826ff4d7a8b0f515e5d8b6d77 with delete_type TABLET_DATA_DELETED (replaced by tablet 657e55fa53c447babfe91b1582806e6b) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.907842 28380 tablet_service.cc:1558] Processing DeleteTablet for tablet 41107eb1d9164a9c98060003b4027412 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f5d4296952e48a48756e6f27140da87) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.908036 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 092a9aba9c6e48f5be9cf5d55ba93678 with delete_type TABLET_DATA_DELETED (replaced by tablet c7f42997416841a7b2c1ad347d2e9282) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.908237 28378 tablet_service.cc:1558] Processing DeleteTablet for tablet bce7bf74b6d24d0bb43808b8a62cf1b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
W20260618 13:59:32.909555 29007 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 038ea0e9c0b04db8b7ec56b065e3cc51 consensus state has no leader
I20260618 13:59:32.909986 29007 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_26e7d8c4f91144268a2a43e83992667d" } new_table_name: "default.Table_f7e1e81727cb4aa8a4e65ddc115b49c0" modify_external_catalogs: true
I20260618 13:59:32.911136 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 7d4144b1b5ea4e968672616bc22bead8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.914810 28525 tablet_service.cc:1558] Processing DeleteTablet for tablet 55ae29c3b7ed44cfa659130745037aa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:31 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:32.915127 29745 tablet_replica.cc:333] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:32.915345 29745 raft_consensus.cc:2243] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:32.915509 29745 raft_consensus.cc:2272] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:32.916064 29745 ts_tablet_manager.cc:1916] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:32.917723 28410 raft_consensus.cc:2468] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:32.918448 28617 leader_election.cc:304] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:32.922828 29719 raft_consensus.cc:2804] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:32.922955 29719 raft_consensus.cc:697] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:32.923180 29719 consensus_queue.cc:237] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:32.927544 29007 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ece7909d056c4ba8a5dca6423f0c1781 consensus state has no leader
I20260618 13:59:32.928609 29007 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_e4b61938ff2f43ba97707a336462a018"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:32.929201 29007 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e4b61938ff2f43ba97707a336462a018 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:32.935273 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet e3d8616846c649b6a969b971db0d03db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:32.935999 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.939941 29756 raft_consensus.cc:493] T 15180aefba0544329daee74308ef5010 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.940104 29756 raft_consensus.cc:515] T 15180aefba0544329daee74308ef5010 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.940721 29756 leader_election.cc:290] T 15180aefba0544329daee74308ef5010 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:32.940596 29007 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_e57bb556500249678a1689b336e0335c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:32.941252 29007 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e57bb556500249678a1689b336e0335c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:32.942489 28380 tablet_service.cc:1558] Processing DeleteTablet for tablet d1baf86497d442dbbbddbc3498bebbee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.942636 28379 tablet_service.cc:1558] Processing DeleteTablet for tablet b51d0851efb34eaa9e9e4be6b292ef62 with delete_type TABLET_DATA_DELETED (replaced by tablet f9f7359c4a144ca89fe3e51d1365a425) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.942668 28378 tablet_service.cc:1558] Processing DeleteTablet for tablet aa8cf927bed9437086efd8e0f21ba26a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.942773 28376 tablet_service.cc:1558] Processing DeleteTablet for tablet e95c88e4b0a046a3ad69ee49af47c7c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.942814 28374 tablet_service.cc:1558] Processing DeleteTablet for tablet fe750fc5d998448caeab0a6c808dfba7 with delete_type TABLET_DATA_DELETED (replaced by tablet 438663f365e34fab87e34f434920cecf) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.942917 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 69d5e75b4a8247b29d69d07fad5ca5db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.943008 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f5a334d18ba4c06bb358fa86d8af80d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.942783 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 7aa9f65d8a5c46fdb745ee17af946dce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.943482 28376 tablet_service.cc:1558] Processing DeleteTablet for tablet d7a97176f72349cda221baaea61d18ad with delete_type TABLET_DATA_DELETED (replaced by tablet 72fd63e8245d4042b091593881ba11b6) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.943841 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet b0285c75c632457383e81805ab036aaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.943903 29759 raft_consensus.cc:515] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:32.948366 28657 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43578: Illegal state: replica d6a169b16ced46ca9cfbc631709bb257 is not leader of this config: current role FOLLOWER
I20260618 13:59:32.949282 29759 leader_election.cc:290] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:32.949537 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet 8fd9fa9675234db2abff694a517c0ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet 538077e459a44ed78c95f2e881b6e0b0) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.951671 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15180aefba0544329daee74308ef5010" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:32.952064 28797 tablet_bootstrap.cc:492] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:32.953521 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9fa9675234db2abff694a517c0ea8" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:32.953708 28683 raft_consensus.cc:3060] T 8fd9fa9675234db2abff694a517c0ea8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.954564 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15180aefba0544329daee74308ef5010" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:32.955076 28797 ts_tablet_manager.cc:1403] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.098s user 0.001s sys 0.002s
I20260618 13:59:32.955876 28797 raft_consensus.cc:359] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.956036 28797 raft_consensus.cc:385] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:32.956099 28797 raft_consensus.cc:740] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:32.956267 28374 tablet_service.cc:1558] Processing DeleteTablet for tablet aef7c1f111634c98af046e73fa9ffdeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.956462 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 751cd9bd961b4f56a588ed02a05f0212 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.956591 28410 raft_consensus.cc:2393] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:32.957297 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet 233040a16b0d4b61b977ca57c1afec2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.957680 28391 tablet_service.cc:1558] Processing DeleteTablet for tablet c4cfca9e76e641a7bc0b58b9ee240f36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.957830 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet d667ea5f029f46d89fcb3be03069b97b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.958112 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet b6dd422c05324c989e29b20ae2afabad with delete_type TABLET_DATA_DELETED (replaced by tablet 97a9f949999446f09359629fe483bf67) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.958655 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b88ed7cf79c4929a8704f66691cbed4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.959024 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet b16f7d3c7837498e996bba1818110642 with delete_type TABLET_DATA_DELETED (replaced by tablet d1baf86497d442dbbbddbc3498bebbee) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.959319 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet c557afc3abfa4ef188bf248e89f49d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.959612 28389 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c75278db2234b11bf95f1cf5cc76edd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.962061 28797 consensus_queue.cc:260] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:32.962582 28797 ts_tablet_manager.cc:1434] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260618 13:59:32.962857 28797 tablet_bootstrap.cc:492] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:32.964489 28797 tablet_bootstrap.cc:654] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:32.966215 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet 55ae29c3b7ed44cfa659130745037aa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:31 UTC) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:32.968662 29025 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.1:48495 (request call id 142) took 2523 ms (client timeout 1000 ms). Trace:
W20260618 13:59:32.968776 29025 rpcz_store.cc:269] 0618 13:59:30.445461 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:30.451973 (+ 6512us) service_pool.cc:224] Handling call
0618 13:59:31.284556 (+832583us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:31.284561 (+ 5us) write_op.cc:454] Released schema lock
0618 13:59:32.968594 (+1684033us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:31.277517 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:31.277652 (+ 135us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:31.277660 (+ 8us) write_op.cc:435] Acquired schema lock
0618 13:59:31.277663 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:31.277771 (+ 108us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:31.277784 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:31.277791 (+ 7us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:31.277864 (+ 73us) tablet.cc:701] Row locks acquired
0618 13:59:31.277866 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:31.277905 (+ 39us) write_op.cc:270] Start()
0618 13:59:31.277940 (+ 35us) write_op.cc:276] Timestamp: P: 1781791171277871 usec, L: 0
0618 13:59:31.277943 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:31.278186 (+ 243us) log.cc:844] Serialized 530 byte log entry
0618 13:59:31.282474 (+ 4288us) op_driver.cc:464] REPLICATION: finished
0618 13:59:31.282620 (+ 146us) write_op.cc:301] APPLY: starting
0618 13:59:31.282639 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:31.282682 (+ 43us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:31.282725 (+ 43us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:31.282784 (+ 59us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:31.282824 (+ 40us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:31.282834 (+ 10us) write_op.cc:312] APPLY: finished
0618 13:59:31.282921 (+ 87us) log.cc:844] Serialized 23 byte log entry
0618 13:59:31.282994 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:31.283017 (+ 23us) write_op.cc:454] Released schema lock
0618 13:59:31.283024 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":34695,"reported_tablets":154,"child_traces":[["op",{"apply.queue_time_us":96,"apply.run_cpu_time_us":469,"apply.run_wall_time_us":468,"num_ops":1,"prepare.queue_time_us":1818,"prepare.run_cpu_time_us":802,"prepare.run_wall_time_us":803,"raft.queue_time_us":969,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":267,"replication_time_us":4497}]]}
I20260618 13:59:32.969795 28663 tablet_service.cc:1558] Processing DeleteTablet for tablet e3d8616846c649b6a969b971db0d03db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:32.971211 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fae928833e44d2ba860d6184d4c0142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.971421 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet d4667fe48f0f4eff9c6c8baabdab58d4 with delete_type TABLET_DATA_DELETED (replaced by tablet a662bb37fb7d4df49e61e524808b67a8) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.971560 28374 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a0039a463ec4a9c82a21a3a69b565fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.972427 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f3d2f236fd44ab8af0524ea282cfa94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.972677 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet ca2ee746c5024cfc9b31c470f9616e7d with delete_type TABLET_DATA_DELETED (replaced by tablet 60b6216474b74a14af6abf977e312b77) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.973528 29023 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e] alter complete (version 2)
I20260618 13:59:32.973551 29020 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet cd06bb7cffc34412b420d70435fce373
I20260618 13:59:32.974128 29025 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_1cd2893d47c743138f1257093bd5b7b7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:32.974813 29025 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1cd2893d47c743138f1257093bd5b7b7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:32.975003 29020 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_82535d8562ad4142860544cc6cbf6fc2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:32.975569 29020 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_82535d8562ad4142860544cc6cbf6fc2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:32.976605 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e25027f423144b088dad955c7282921 with delete_type TABLET_DATA_DELETED (replaced by tablet a8da96fc621c47d4a444b5a39259f85d) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.976823 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 18d8b609b96a4e81b57f69bff522eeea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.977661 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd9fa9675234db2abff694a517c0ea8" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:32.977833 28410 raft_consensus.cc:3060] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:32.978634 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet cd06bb7cffc34412b420d70435fce373 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:57282
W20260618 13:59:32.980389 29019 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ab100ebbe61a4321b9e76c730e5d46ed consensus state has no leader
I20260618 13:59:32.980769 29019 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_ce0dbbee92c44d35b8deaf3d68303529"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:32.981159 28483 leader_election.cc:304] T 15180aefba0544329daee74308ef5010 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:32.981292 29019 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ce0dbbee92c44d35b8deaf3d68303529 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:32.981858 28617 consensus_peers.cc:597] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f278d0acbb0c4a0ca61640bc142e6335. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:32.982573 29714 consensus_queue.cc:1048] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:32.984004 29007 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_b2262e1c515848f7b545886563ac854a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:32.984143 29759 raft_consensus.cc:3060] T 15180aefba0544329daee74308ef5010 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:32.984681 29007 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b2262e1c515848f7b545886563ac854a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:32.985265 29679 raft_consensus.cc:493] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:32.985430 29679 raft_consensus.cc:515] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:32.985954 29679 leader_election.cc:290] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:32.986253 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 97a9f949999446f09359629fe483bf67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:29 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.986603 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet e5f7eaa7d218440ca2c35340c5c87814 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.986874 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 3dd0eaa64271489d92aa380df877f257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.987195 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 04dac931e62a448d99633878eb70b92f with delete_type TABLET_DATA_DELETED (replaced by tablet 15483938a06e459789f87c6bb456bd15) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.987494 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet ab521028ac1a4d0da013112d76bb169f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.987762 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet f3ba9186b3ff47d5a2b158fef619686e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.988346 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet c3d523419ff244f085d6eac1a3231973 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.988687 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 7572d17724d249739be1a1cb9468f909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:27 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.989050 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 015494374645469785f1f978eaf0e9a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 2e493f647a344cd581515a77230d765f) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.989563 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 75824a5d2f7e47838b60e6a2b785c6b2 with delete_type TABLET_DATA_DELETED (replaced by tablet d4539f767c054e9e8ebc1f3f56108e7e) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.989920 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet be59f988f0e34b9480f859cf88b9c223 with delete_type TABLET_DATA_DELETED (replaced by tablet b968a8d4014341ca9c5964263721ac79) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.990270 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 0463fa2e30554abcbfc8f37b702e81d4 with delete_type TABLET_DATA_DELETED (replaced by tablet fe1139c409e84070a2f496a85545e8d5) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.990659 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 6eea4d59935d40aa97dc10ec7469aa56 with delete_type TABLET_DATA_DELETED (replaced by tablet 4bc87770baa34842a25ed97a19354446) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.991008 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 55ae29c3b7ed44cfa659130745037aa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:31 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:32.993490 29400 consensus_queue.cc:1048] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:32.993963 28458 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:32.994284 28798 ts_tablet_manager.cc:1434] T 15180aefba0544329daee74308ef5010 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.155s user 0.001s sys 0.012s
I20260618 13:59:32.996562 28798 tablet_bootstrap.cc:492] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:33.001178 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 02b5cc08ca0c4e719e0414b345211ba5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:33.003243 28798 tablet_bootstrap.cc:654] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.004791 29679 raft_consensus.cc:493] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.004927 29679 raft_consensus.cc:515] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.005561 29400 raft_consensus.cc:493] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.005587 29679 leader_election.cc:290] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:33.005676 29400 raft_consensus.cc:515] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.006188 29400 leader_election.cc:290] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:33.006848 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72fd63e8245d4042b091593881ba11b6" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:33.007047 28408 raft_consensus.cc:2468] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:33.007509 28617 leader_election.cc:304] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:33.007792 29400 raft_consensus.cc:2804] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:33.007854 29400 raft_consensus.cc:493] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:33.007905 29400 raft_consensus.cc:3060] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.009459 29746 raft_consensus.cc:2243] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:33.009892 29746 raft_consensus.cc:2272] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:33.009919 28545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72fd63e8245d4042b091593881ba11b6" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:33.012336 28618 leader_election.cc:343] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.013484 28663 tablet_service.cc:1558] Processing DeleteTablet for tablet cd06bb7cffc34412b420d70435fce373 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:33.014147 28381 tablet_service.cc:1558] Processing DeleteTablet for tablet e6de1293572f4e75af3417f281b9a142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:33.014364 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 827cc0987bd541759e334fb620f04ed5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:33.014880 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet bc0adedecee4405e86b0a37b9e62e352 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:33.015193 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet cd06bb7cffc34412b420d70435fce373 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:33.016786 28385 tablet_service.cc:1558] Processing DeleteTablet for tablet 591465e2b57247a2b224cc8b09225811 with delete_type TABLET_DATA_DELETED (replaced by tablet d4667fe48f0f4eff9c6c8baabdab58d4) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:33.016957 29746 ts_tablet_manager.cc:1916] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:33.019706 29566 ts_tablet_manager.cc:1929] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:33.019811 29566 log.cc:1199] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/897bf6d6bbee4ad89c296325d94a3e53
I20260618 13:59:33.020155 29566 ts_tablet_manager.cc:1950] T 897bf6d6bbee4ad89c296325d94a3e53 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:33.020426 29303 raft_consensus.cc:515] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.021049 29303 leader_election.cc:290] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:33.023995 28683 raft_consensus.cc:2468] T 8fd9fa9675234db2abff694a517c0ea8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:33.027792 29644 consensus_queue.cc:1048] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:33.029953 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b51d0851efb34eaa9e9e4be6b292ef62" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:33.030151 28408 raft_consensus.cc:3060] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.031404 28410 raft_consensus.cc:2468] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:33.032075 28483 leader_election.cc:304] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2; no voters:
I20260618 13:59:33.032380 29756 raft_consensus.cc:2804] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:33.032495 29756 raft_consensus.cc:697] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:33.032752 29756 consensus_queue.cc:237] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.035449 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 23) took 2157 ms. Trace:
I20260618 13:59:33.035519 29566 rpcz_store.cc:276] 0618 13:59:30.877558 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:30.878025 (+ 467us) service_pool.cc:224] Handling call
0618 13:59:32.497975 (+1619950us) ts_tablet_manager.cc:1151] Deleting tablet 897bf6d6bbee4ad89c296325d94a3e53
0618 13:59:32.497985 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:32.823987 (+326002us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:33.019629 (+195642us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:33.035430 (+ 15801us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1608641}
I20260618 13:59:33.035928 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 42) took 1054 ms. Trace:
I20260618 13:59:33.036002 29566 rpcz_store.cc:276] 0618 13:59:31.981063 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:31.981126 (+ 63us) service_pool.cc:224] Handling call
0618 13:59:33.035790 (+1054664us) ts_tablet_manager.cc:1151] Deleting tablet c4cfca9e76e641a7bc0b58b9ee240f36
0618 13:59:33.035801 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:33.035909 (+ 108us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1054453}
I20260618 13:59:33.036243 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 44) took 1054 ms. Trace:
I20260618 13:59:33.036291 29566 rpcz_store.cc:276] 0618 13:59:31.981272 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:31.981318 (+ 46us) service_pool.cc:224] Handling call
0618 13:59:33.036170 (+1054852us) ts_tablet_manager.cc:1151] Deleting tablet e5f7eaa7d218440ca2c35340c5c87814
0618 13:59:33.036178 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:33.036234 (+ 56us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1054711}
I20260618 13:59:33.036527 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 43) took 1055 ms. Trace:
I20260618 13:59:33.036576 29566 rpcz_store.cc:276] 0618 13:59:31.981180 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:31.981585 (+ 405us) service_pool.cc:224] Handling call
0618 13:59:33.036459 (+1054874us) ts_tablet_manager.cc:1151] Deleting tablet 3dd0eaa64271489d92aa380df877f257
0618 13:59:33.036465 (+ 6us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:33.036517 (+ 52us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1054708}
I20260618 13:59:33.036785 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 45) took 1014 ms. Trace:
I20260618 13:59:33.036834 29566 rpcz_store.cc:276] 0618 13:59:32.022141 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.022206 (+ 65us) service_pool.cc:224] Handling call
0618 13:59:33.036720 (+1014514us) ts_tablet_manager.cc:1151] Deleting tablet 97a9f949999446f09359629fe483bf67
0618 13:59:33.036726 (+ 6us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:33.036775 (+ 49us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1014318}
I20260618 13:59:33.037027 29566 tablet_replica.cc:333] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:33.037192 29566 raft_consensus.cc:2243] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:33.037467 29566 raft_consensus.cc:2272] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:33.037950 29566 ts_tablet_manager.cc:1916] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:33.056684 28983 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.60:42083, user_credentials={real_user=slave}} blocked reactor thread for 126153us
I20260618 13:59:33.057080 28545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b51d0851efb34eaa9e9e4be6b292ef62" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:33.059166 28618 leader_election.cc:343] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.059616 29679 raft_consensus.cc:493] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.059782 29679 raft_consensus.cc:515] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.060279 29679 leader_election.cc:290] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:33.068791 28618 consensus_peers.cc:597] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:33.070942 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 897bf6d6bbee4ad89c296325d94a3e53 (table default.Table_35260f90f6814fa3915b4280cff96d1b [id=02ffed3d852845d6845fb5176f036fce]) successfully deleted
I20260618 13:59:33.071164 29303 raft_consensus.cc:493] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.071298 29303 raft_consensus.cc:515] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:33.071395 28989 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet c4cfca9e76e641a7bc0b58b9ee240f36 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c4cfca9e76e641a7bc0b58b9ee240f36 already in progress: creating tablet
W20260618 13:59:33.071624 28989 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet e5f7eaa7d218440ca2c35340c5c87814 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5f7eaa7d218440ca2c35340c5c87814 already in progress: creating tablet
W20260618 13:59:33.071817 28989 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 3dd0eaa64271489d92aa380df877f257 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3dd0eaa64271489d92aa380df877f257 already in progress: creating tablet
W20260618 13:59:33.072019 28989 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 97a9f949999446f09359629fe483bf67 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 97a9f949999446f09359629fe483bf67 already in progress: creating tablet
I20260618 13:59:33.072381 29303 leader_election.cc:290] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:33.072672 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c881dd3cb66344969310956ad6e83cc0" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:33.072877 28410 raft_consensus.cc:2468] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:33.073400 28617 leader_election.cc:304] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:33.073793 29303 raft_consensus.cc:2804] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:33.073894 29303 raft_consensus.cc:493] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:33.073972 29303 raft_consensus.cc:3060] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.074797 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c881dd3cb66344969310956ad6e83cc0" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:33.076125 28618 leader_election.cc:343] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.082171 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316110bd2d024c7599fa2933d87e559e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:33.082216 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316110bd2d024c7599fa2933d87e559e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:33.082934 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72fd63e8245d4042b091593881ba11b6" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:33.083695 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72fd63e8245d4042b091593881ba11b6" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:33.083873 28346 consensus_peers.cc:597] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b -> Peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Couldn't send request to peer d6a169b16ced46ca9cfbc631709bb257. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 41107eb1d9164a9c98060003b4027412. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:33.084020 28346 leader_election.cc:343] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.084502 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18d8b609b96a4e81b57f69bff522eeea" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:33.084534 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18d8b609b96a4e81b57f69bff522eeea" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:33.085222 28346 leader_election.cc:343] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.085337 28346 leader_election.cc:304] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:33.085613 29679 raft_consensus.cc:2749] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:33.085670 28346 leader_election.cc:343] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.085844 28797 tablet_bootstrap.cc:492] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
W20260618 13:59:33.085812 28346 leader_election.cc:343] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.085968 28797 ts_tablet_manager.cc:1403] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.123s user 0.010s sys 0.006s
W20260618 13:59:33.085999 28346 leader_election.cc:343] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.086084 28346 leader_election.cc:304] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:33.086270 29679 raft_consensus.cc:2749] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:33.089787 29714 raft_consensus.cc:493] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.089959 29714 raft_consensus.cc:515] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.090523 29714 leader_election.cc:290] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:33.090957 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f42997416841a7b2c1ad347d2e9282" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:33.090986 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f42997416841a7b2c1ad347d2e9282" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:33.091176 28403 raft_consensus.cc:2468] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:33.091691 28617 leader_election.cc:304] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:33.092053 29714 raft_consensus.cc:2804] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:33.092157 29714 raft_consensus.cc:493] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:33.092206 29714 raft_consensus.cc:3060] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.089941 28797 raft_consensus.cc:359] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.092795 28797 raft_consensus.cc:385] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.092913 28797 raft_consensus.cc:740] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.093142 28797 consensus_queue.cc:260] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:33.092881 28618 leader_election.cc:343] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.094795 28797 ts_tablet_manager.cc:1434] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260618 13:59:33.095136 28797 tablet_bootstrap.cc:492] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:33.098207 28618 consensus_peers.cc:597] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:33.098834 28797 tablet_bootstrap.cc:654] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.104820 29679 raft_consensus.cc:493] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.104995 29679 raft_consensus.cc:515] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.105652 29679 leader_election.cc:290] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:33.106029 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54910015987484e95b295352d859315" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:33.106356 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54910015987484e95b295352d859315" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:33.107062 28346 leader_election.cc:343] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:33.107316 28346 leader_election.cc:343] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.107447 28346 leader_election.cc:304] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:33.107719 29679 raft_consensus.cc:2749] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:33.125626 29782 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:43578 (request call id 63) took 1076 ms. Trace:
I20260618 13:59:33.125758 29782 rpcz_store.cc:276] 0618 13:59:32.049567 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.049616 (+ 49us) service_pool.cc:224] Handling call
0618 13:59:33.125604 (+1075988us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:32.050040 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0618 13:59:32.050146 (+ 106us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0618 13:59:32.050156 (+ 10us) alter_schema_op.cc:66] Acquired schema lock
0618 13:59:32.050163 (+ 7us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0618 13:59:32.050205 (+ 42us) alter_schema_op.cc:128] START. Timestamp: 1781791172050148
0618 13:59:32.050212 (+ 7us) op_driver.cc:348] REPLICATION: starting
0618 13:59:32.050482 (+ 270us) log.cc:844] Serialized 173 byte log entry
0618 13:59:32.362614 (+312132us) op_driver.cc:464] REPLICATION: finished
0618 13:59:32.547653 (+185039us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0618 13:59:32.548135 (+ 482us) log.cc:844] Serialized 5 byte log entry
0618 13:59:33.125280 (+577145us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:33.125525 (+ 245us) log.cc:844] Serialized 14 byte log entry
0618 13:59:33.125564 (+ 39us) alter_schema_op.cc:73] Released schema lock
0618 13:59:33.125569 (+ 5us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":184986,"prepare.queue_time_us":57,"prepare.run_cpu_time_us":9664,"prepare.run_wall_time_us":50160,"replication_time_us":312368,"thread_start_us":49581,"threads_started":1,"wal-append.queue_time_us":49138}]]}
I20260618 13:59:33.126600 29400 raft_consensus.cc:515] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.127311 29400 leader_election.cc:290] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:33.127490 28408 raft_consensus.cc:2468] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:33.127779 29745 ts_tablet_manager.cc:1929] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:33.127885 29745 log.cc:1199] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/d3dbf13544ce408ab97834aec5862390
I20260618 13:59:33.128193 29745 ts_tablet_manager.cc:1950] T d3dbf13544ce408ab97834aec5862390 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:33.128973 28683 raft_consensus.cc:2393] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6fc808af0e884ed8836561665d8e0e7b in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:33.130563 28617 leader_election.cc:304] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:33.130959 29400 raft_consensus.cc:2804] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:33.131052 29400 raft_consensus.cc:697] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:33.131242 29400 consensus_queue.cc:237] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.132215 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72fd63e8245d4042b091593881ba11b6" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:33.132412 28408 raft_consensus.cc:3060] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.136931 29746 ts_tablet_manager.cc:1929] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:33.137043 29746 log.cc:1199] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/d3dbf13544ce408ab97834aec5862390
I20260618 13:59:33.137389 29746 ts_tablet_manager.cc:1950] T d3dbf13544ce408ab97834aec5862390 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:33.145452 28799 tablet_bootstrap.cc:492] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:33.145599 28799 ts_tablet_manager.cc:1403] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.255s user 0.029s sys 0.023s
I20260618 13:59:33.146284 28799 raft_consensus.cc:359] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.146492 28799 raft_consensus.cc:385] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.146548 28799 raft_consensus.cc:740] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.146727 28799 consensus_queue.cc:260] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.147233 28799 ts_tablet_manager.cc:1434] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:33.147502 28799 tablet_bootstrap.cc:492] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:33.149264 28799 tablet_bootstrap.cc:654] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.150120 29020 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_aab2bb1a1fc6434b9720650bdc13c115"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:33.150254 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72fd63e8245d4042b091593881ba11b6" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:33.150810 29020 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aab2bb1a1fc6434b9720650bdc13c115 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:33.150979 28618 leader_election.cc:343] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.152282 29025 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_39e2396806924281b1b78edd2295809b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:33.152767 29025 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_39e2396806924281b1b78edd2295809b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:33.155058 29755 raft_consensus.cc:2749] T 118259402d7d48dda381cdd18eec0f1b P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:33.155459 28346 leader_election.cc:304] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:33.159238 29019 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_dba55442642c45b1b8dfe959d62699c9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:33.159791 29019 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dba55442642c45b1b8dfe959d62699c9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:33.161095 29007 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_7c9d04c2f42d481e9ec6c499648cd688"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:33.161574 29007 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7c9d04c2f42d481e9ec6c499648cd688 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:33.165057 28798 tablet_bootstrap.cc:492] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:33.167433 28798 ts_tablet_manager.cc:1403] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.171s user 0.014s sys 0.022s
I20260618 13:59:33.168509 28798 raft_consensus.cc:359] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.174564 28797 tablet_bootstrap.cc:492] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:33.174707 28797 ts_tablet_manager.cc:1403] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.080s user 0.008s sys 0.000s
I20260618 13:59:33.174499 29456 raft_consensus.cc:2749] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:33.175243 28798 raft_consensus.cc:385] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.175468 28797 raft_consensus.cc:359] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.175554 28798 raft_consensus.cc:740] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.175633 28797 raft_consensus.cc:385] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.175697 28797 raft_consensus.cc:740] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.176029 28798 consensus_queue.cc:260] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.177399 28798 ts_tablet_manager.cc:1434] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260618 13:59:33.178229 28798 tablet_bootstrap.cc:492] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:33.180575 28730 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:33.180788 28799 tablet_bootstrap.cc:492] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:33.180917 28799 ts_tablet_manager.cc:1403] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.005s
I20260618 13:59:33.181694 28799 raft_consensus.cc:359] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.181860 28799 raft_consensus.cc:385] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.181919 28799 raft_consensus.cc:740] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.182137 28799 consensus_queue.cc:260] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.189004 28798 tablet_bootstrap.cc:654] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.200241 28799 ts_tablet_manager.cc:1434] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.019s user 0.001s sys 0.005s
I20260618 13:59:33.202003 28799 tablet_bootstrap.cc:492] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:33.202935 28544 raft_consensus.cc:3060] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.204694 28799 tablet_bootstrap.cc:654] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.175881 28797 consensus_queue.cc:260] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.217674 28408 raft_consensus.cc:2468] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:33.222087 29303 raft_consensus.cc:515] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.222733 29303 leader_election.cc:290] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:33.225970 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c881dd3cb66344969310956ad6e83cc0" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:33.226886 29745 tablet_replica.cc:333] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:33.227205 29745 raft_consensus.cc:2243] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:33.227396 29745 raft_consensus.cc:2272] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:33.228087 29745 ts_tablet_manager.cc:1916] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:33.228578 28618 leader_election.cc:343] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:33.229020 28346 consensus_peers.cc:597] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:33.230574 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet d3dbf13544ce408ab97834aec5862390 (table default.Table_bf5ce6c6e0b64002994fb45e76ab30eb [id=a0e6904d88704db39c194c34ac9a7faf]) successfully deleted
W20260618 13:59:33.230840 28988 catalog_manager.cc:5007] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet e3d8616846c649b6a969b971db0d03db because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e3d8616846c649b6a969b971db0d03db already in progress: creating tablet
I20260618 13:59:33.231069 29679 raft_consensus.cc:2749] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:33.232597 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c881dd3cb66344969310956ad6e83cc0" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:33.232779 28411 raft_consensus.cc:3060] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.234182 28617 leader_election.cc:304] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:33.234603 29644 raft_consensus.cc:2804] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:33.234714 29644 raft_consensus.cc:697] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:33.234931 29644 consensus_queue.cc:237] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.237668 29400 consensus_queue.cc:1048] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:33.243396 29566 ts_tablet_manager.cc:1929] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:33.243649 29566 log.cc:1199] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/d9c00003300246ada2a356e3e1cf684b
I20260618 13:59:33.244233 29566 ts_tablet_manager.cc:1950] T d9c00003300246ada2a356e3e1cf684b P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:33.246559 28797 ts_tablet_manager.cc:1434] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.072s user 0.002s sys 0.000s
I20260618 13:59:33.247128 28797 tablet_bootstrap.cc:492] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:33.258234 28988 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet 15180aefba0544329daee74308ef5010 (table default.Table_97b6bb39b3bf4612be391b9dd78d1646 [id=42a2a24590834abc9537669aaf223a7c], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 15180aefba0544329daee74308ef5010 consensus state has no leader
I20260618 13:59:33.266736 29007 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_c99584ecc88948a097b39c22d68b3938" } modify_external_catalogs: true
I20260618 13:59:33.266973 29007 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_c99584ecc88948a097b39c22d68b3938" } modify_external_catalogs: true
I20260618 13:59:33.268440 29019 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_efb2b8e451084b7ba0de07e0a6e538c2" } new_table_name: "default.Table_c70d6c5b0dc94b33b4c5772edd2b9dca" modify_external_catalogs: true
W20260618 13:59:33.271212 28983 consensus_peers.cc:597] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 -> Peer 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083): Couldn't send request to peer 7637285570574a6dbcef1de6f84ecac0. Status: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260618 13:59:33.271993 28618 consensus_peers.cc:597] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:33.273600 29811 tablet.cc:1724] T 512a9e63ba324a30b8d8c37e585ebdf8 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:33.249938 28797 tablet_bootstrap.cc:654] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.280792 28524 tablet_service.cc:1511] Processing CreateTablet for tablet ba18e879631947f88bd266681746ec82 (DEFAULT_TABLE table=default.Table_bf5ce6c6e0b64002994fb45e76ab30eb [id=a0e6904d88704db39c194c34ac9a7faf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.281208 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba18e879631947f88bd266681746ec82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.281891 28798 tablet_bootstrap.cc:492] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:33.282351 28798 ts_tablet_manager.cc:1403] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.104s user 0.009s sys 0.018s
I20260618 13:59:33.285233 28659 tablet_service.cc:1511] Processing CreateTablet for tablet ba18e879631947f88bd266681746ec82 (DEFAULT_TABLE table=default.Table_bf5ce6c6e0b64002994fb45e76ab30eb [id=a0e6904d88704db39c194c34ac9a7faf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.285635 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba18e879631947f88bd266681746ec82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.286542 28798 raft_consensus.cc:359] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.288092 28798 raft_consensus.cc:385] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.288208 28798 raft_consensus.cc:740] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.288954 28799 tablet_bootstrap.cc:492] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:33.289141 28798 consensus_queue.cc:260] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.289438 28799 ts_tablet_manager.cc:1403] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.088s user 0.012s sys 0.012s
I20260618 13:59:33.289470 29782 tablet.cc:1724] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:33.290182 28798 ts_tablet_manager.cc:1434] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260618 13:59:33.290467 28798 tablet_bootstrap.cc:492] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:33.292713 28522 tablet_service.cc:1511] Processing CreateTablet for tablet b89b26d1595642ff82381f8baddfb99a (DEFAULT_TABLE table=default.Table_19806c50abb24f379c7afe2107b88b7f [id=5a913492c55f4bf7b9d1b3bcc614ae19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.291807 28799 raft_consensus.cc:359] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.293280 28652 tablet_service.cc:1511] Processing CreateTablet for tablet b89b26d1595642ff82381f8baddfb99a (DEFAULT_TABLE table=default.Table_19806c50abb24f379c7afe2107b88b7f [id=5a913492c55f4bf7b9d1b3bcc614ae19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.294287 28798 tablet_bootstrap.cc:654] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.294481 28799 raft_consensus.cc:385] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.293308 29775 tablet.cc:1724] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:33.294551 28799 raft_consensus.cc:740] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.294749 28385 tablet_service.cc:1511] Processing CreateTablet for tablet b89b26d1595642ff82381f8baddfb99a (DEFAULT_TABLE table=default.Table_19806c50abb24f379c7afe2107b88b7f [id=5a913492c55f4bf7b9d1b3bcc614ae19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.294729 28799 consensus_queue.cc:260] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.294123 28388 tablet_service.cc:1511] Processing CreateTablet for tablet ba18e879631947f88bd266681746ec82 (DEFAULT_TABLE table=default.Table_bf5ce6c6e0b64002994fb45e76ab30eb [id=a0e6904d88704db39c194c34ac9a7faf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.296036 28799 ts_tablet_manager.cc:1434] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260618 13:59:33.296305 28799 tablet_bootstrap.cc:492] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:33.298965 28618 consensus_peers.cc:597] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:33.300397 28799 tablet_bootstrap.cc:654] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.303989 28798 tablet_bootstrap.cc:492] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:33.304138 28798 ts_tablet_manager.cc:1403] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20260618 13:59:33.305078 28798 raft_consensus.cc:359] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.305289 28798 raft_consensus.cc:385] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.308584 29755 raft_consensus.cc:493] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.308758 29755 raft_consensus.cc:515] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.309358 29755 leader_election.cc:290] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:33.310101 28406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6de1293572f4e75af3417f281b9a142" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:33.310308 28406 raft_consensus.cc:2393] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:33.310956 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6de1293572f4e75af3417f281b9a142" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:33.311183 28798 raft_consensus.cc:740] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.312279 28798 consensus_queue.cc:260] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.312923 28484 leader_election.cc:304] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:33.313336 29755 raft_consensus.cc:3060] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.314586 29679 consensus_queue.cc:1048] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:33.308569 28797 tablet_bootstrap.cc:492] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:33.315276 28797 ts_tablet_manager.cc:1403] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.068s user 0.004s sys 0.000s
I20260618 13:59:33.316033 28797 raft_consensus.cc:359] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.316263 28797 raft_consensus.cc:385] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.316359 28797 raft_consensus.cc:740] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.316582 28797 consensus_queue.cc:260] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.317101 28797 ts_tablet_manager.cc:1434] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:33.317454 28797 tablet_bootstrap.cc:492] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:33.318925 28411 raft_consensus.cc:2468] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:33.318941 29714 raft_consensus.cc:515] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.319252 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b89b26d1595642ff82381f8baddfb99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.319562 29714 leader_election.cc:290] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:33.319819 29749 consensus_meta.cc:287] T c0a42aa779ae452e9420d730bf8d7d55 P 97859748af2c41879ba2a668dda999d2: Time spent flushing consensus metadata: real 0.535s user 0.003s sys 0.000s
I20260618 13:59:33.319917 29749 raft_consensus.cc:2749] T c0a42aa779ae452e9420d730bf8d7d55 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:33.325497 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b89b26d1595642ff82381f8baddfb99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.326264 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba18e879631947f88bd266681746ec82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.328372 28799 tablet_bootstrap.cc:492] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:33.328505 28799 ts_tablet_manager.cc:1403] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.002s
I20260618 13:59:33.329303 28799 raft_consensus.cc:359] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.329474 28799 raft_consensus.cc:385] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.329522 28799 raft_consensus.cc:740] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.330597 28798 ts_tablet_manager.cc:1434] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.026s user 0.003s sys 0.000s
I20260618 13:59:33.330864 28798 tablet_bootstrap.cc:492] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:33.332592 28798 tablet_bootstrap.cc:654] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.333470 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 46) took 1310 ms. Trace:
I20260618 13:59:33.335942 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet d3dbf13544ce408ab97834aec5862390 (table default.Table_bf5ce6c6e0b64002994fb45e76ab30eb [id=a0e6904d88704db39c194c34ac9a7faf]) successfully deleted
W20260618 13:59:33.336266 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 55ae29c3b7ed44cfa659130745037aa6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55ae29c3b7ed44cfa659130745037aa6 already in progress: creating tablet
W20260618 13:59:33.336498 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet e3d8616846c649b6a969b971db0d03db because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e3d8616846c649b6a969b971db0d03db already in progress: creating tablet
W20260618 13:59:33.336746 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet cd06bb7cffc34412b420d70435fce373 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd06bb7cffc34412b420d70435fce373 already in progress: creating tablet
I20260618 13:59:33.337649 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f42997416841a7b2c1ad347d2e9282" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:33.337984 29644 consensus_queue.cc:1048] T c0a42aa779ae452e9420d730bf8d7d55 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:33.338200 28618 leader_election.cc:343] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.340126 29811 tablet.cc:1724] T ece7909d056c4ba8a5dca6423f0c1781 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:33.341693 28799 consensus_queue.cc:260] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.344100 28799 ts_tablet_manager.cc:1434] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.016s user 0.001s sys 0.001s
I20260618 13:59:33.344952 29782 tablet.cc:1719] T ece7909d056c4ba8a5dca6423f0c1781 P d6a169b16ced46ca9cfbc631709bb257: Skipping requested alter to schema version 2, tablet already version 2
I20260618 13:59:33.348541 28798 tablet_bootstrap.cc:492] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:33.349313 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f42997416841a7b2c1ad347d2e9282" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:33.349481 28798 ts_tablet_manager.cc:1403] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.000s
I20260618 13:59:33.349732 28408 raft_consensus.cc:3060] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.350207 28798 raft_consensus.cc:359] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.350365 28798 raft_consensus.cc:385] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.333546 29566 rpcz_store.cc:276] 0618 13:59:32.023121 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.023179 (+ 58us) service_pool.cc:224] Handling call
0618 13:59:33.036992 (+1013813us) ts_tablet_manager.cc:1151] Deleting tablet d9c00003300246ada2a356e3e1cf684b
0618 13:59:33.036999 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:33.218592 (+181593us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:33.243296 (+ 24704us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:33.333443 (+ 90147us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1013662}
I20260618 13:59:33.352463 29566 tablet_replica.cc:333] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:33.352581 28798 raft_consensus.cc:740] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.352638 29566 raft_consensus.cc:2243] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:33.352895 29566 raft_consensus.cc:2272] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:33.353124 28798 consensus_queue.cc:260] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.353384 29566 ts_tablet_manager.cc:1916] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:33.354223 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet d9c00003300246ada2a356e3e1cf684b (table default.Table_a9703f9d3b54453691c354131f824e22 [id=f298b441bd9b444b9edd832be0adb03a]) successfully deleted
I20260618 13:59:33.355422 28798 ts_tablet_manager.cc:1434] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
W20260618 13:59:33.356060 28053 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 54729.6us
W20260618 13:59:33.356341 28618 consensus_peers.cc:597] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:33.356899 29679 raft_consensus.cc:493] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.357028 29679 raft_consensus.cc:515] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.357271 28652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b89b26d1595642ff82381f8baddfb99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.341471 28617 leader_election.cc:304] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:33.357597 29679 leader_election.cc:290] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:33.358042 29714 raft_consensus.cc:2804] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:33.358155 29714 raft_consensus.cc:697] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:33.358601 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be00bfa9bc1a4b83b18bfad05ba1495e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:33.359077 28346 leader_election.cc:343] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.360167 29714 consensus_queue.cc:237] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.360934 28799 tablet_bootstrap.cc:492] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:33.360822 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be00bfa9bc1a4b83b18bfad05ba1495e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:33.362772 28346 leader_election.cc:343] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.362886 28346 leader_election.cc:304] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:33.363356 29679 raft_consensus.cc:2749] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:33.364552 28799 tablet_bootstrap.cc:654] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.367669 28798 tablet_bootstrap.cc:492] T 39b06ae7444a48dbbec9e096fd7e4052 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:33.373504 28798 tablet_bootstrap.cc:654] T 39b06ae7444a48dbbec9e096fd7e4052 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.375731 28799 tablet_bootstrap.cc:492] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:33.375870 28799 ts_tablet_manager.cc:1403] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.000s
I20260618 13:59:33.378238 28799 raft_consensus.cc:359] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.378441 28799 raft_consensus.cc:385] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.379387 28798 tablet_bootstrap.cc:492] T 39b06ae7444a48dbbec9e096fd7e4052 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:33.379523 28798 ts_tablet_manager.cc:1403] T 39b06ae7444a48dbbec9e096fd7e4052 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20260618 13:59:33.380318 28798 raft_consensus.cc:359] T 39b06ae7444a48dbbec9e096fd7e4052 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.380504 28798 raft_consensus.cc:385] T 39b06ae7444a48dbbec9e096fd7e4052 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.380563 28798 raft_consensus.cc:740] T 39b06ae7444a48dbbec9e096fd7e4052 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.378623 29679 raft_consensus.cc:493] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.381421 29679 raft_consensus.cc:515] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.382082 29679 leader_election.cc:290] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:33.382684 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be59f988f0e34b9480f859cf88b9c223" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:33.382858 28681 raft_consensus.cc:2468] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:33.382906 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be59f988f0e34b9480f859cf88b9c223" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:33.383374 28346 leader_election.cc:343] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.383572 28346 leader_election.cc:304] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:33.383934 29679 raft_consensus.cc:2804] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:33.384008 29679 raft_consensus.cc:493] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:33.384090 29679 raft_consensus.cc:3060] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.378508 28799 raft_consensus.cc:740] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.387472 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet f45556a59c424ca2bc30956566374c89 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:33.387831 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:33.388192 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:33.387884 28799 consensus_queue.cc:260] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.388959 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:33.389426 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:33.390048 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:33.390177 28799 ts_tablet_manager.cc:1434] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.014s user 0.003s sys 0.000s
I20260618 13:59:33.390367 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:33.390633 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:33.391203 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_efb2b8e451084b7ba0de07e0a6e538c2 [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 1 got 0
I20260618 13:59:33.391420 28799 tablet_bootstrap.cc:492] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:33.391729 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:33.392264 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:33.392553 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 15180aefba0544329daee74308ef5010 (table default.Table_97b6bb39b3bf4612be391b9dd78d1646 [id=42a2a24590834abc9537669aaf223a7c]). Expected version 1 got 0
I20260618 13:59:33.392561 28797 tablet_bootstrap.cc:654] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.393051 28798 consensus_queue.cc:260] T 39b06ae7444a48dbbec9e096fd7e4052 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.394968 28798 ts_tablet_manager.cc:1434] T 39b06ae7444a48dbbec9e096fd7e4052 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260618 13:59:33.395197 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:33.395746 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:33.395944 29872 raft_consensus.cc:493] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.396067 29872 raft_consensus.cc:515] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.396754 29872 leader_election.cc:290] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:33.397485 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "657e55fa53c447babfe91b1582806e6b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:33.396751 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:33.397634 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "657e55fa53c447babfe91b1582806e6b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:33.397905 28346 leader_election.cc:343] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:33.398324 28346 leader_election.cc:343] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.398448 28346 leader_election.cc:304] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:33.398793 29872 raft_consensus.cc:2749] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:33.400053 28798 tablet_bootstrap.cc:492] T 72fd63e8245d4042b091593881ba11b6 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:33.400342 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:33.400604 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 512a9e63ba324a30b8d8c37e585ebdf8 (table default.Table_c70bc1c98b3b49d7aa3080904667732d [id=901874cb80e74ba589679bb7b95c8fe4]). Expected version 1 got 0
I20260618 13:59:33.401206 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:33.401698 29020 catalog_manager.cc:5696] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2 reported cstate change: term changed from 1 to 2, leader changed from d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3) to 97859748af2c41879ba2a668dda999d2 (127.27.100.2). New cstate: current_term: 2 leader_uuid: "97859748af2c41879ba2a668dda999d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:33.402065 28799 tablet_bootstrap.cc:654] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.404057 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet dc36fbea853a4d4e9f61909200239aed (table default.Table_dbc6162c06f74e71b792bd1f7b2388fe [id=49576dabdecf41aca1b15ff3fc0c113c]). Expected version 6 got 0
I20260618 13:59:33.404556 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet ece7909d056c4ba8a5dca6423f0c1781 (table default.Table_f7e1e81727cb4aa8a4e65ddc115b49c0 [id=803d5220ce1a4dcfaf27d93c574be2ac]). Expected version 2 got 0
I20260618 13:59:33.405254 28798 tablet_bootstrap.cc:654] T 72fd63e8245d4042b091593881ba11b6 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.410816 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:33.411250 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:33.411790 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:33.412365 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a]). Expected version 1 got 0
I20260618 13:59:33.412824 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 39b06ae7444a48dbbec9e096fd7e4052 (table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15]). Expected version 2 got 0
I20260618 13:59:33.413149 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:33.415091 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:33.415349 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet cea4bc13eabc4dd98b4f96723498014c (table default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c]). Expected version 1 got 0
I20260618 13:59:33.415702 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:33.415977 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_7f787ab851be496dae97cba8835db19b [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 1 got 0
I20260618 13:59:33.416297 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:33.416584 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:33.416868 29020 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:33.420333 29759 raft_consensus.cc:2749] T 15180aefba0544329daee74308ef5010 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:33.420712 29679 raft_consensus.cc:515] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.421443 29679 leader_election.cc:290] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:33.430009 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be59f988f0e34b9480f859cf88b9c223" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:33.430230 29719 consensus_queue.cc:1048] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:33.430548 28346 leader_election.cc:343] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.432957 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be59f988f0e34b9480f859cf88b9c223" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:33.433145 28683 raft_consensus.cc:3060] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.438129 29775 tablet.cc:1719] T ece7909d056c4ba8a5dca6423f0c1781 P 6fc808af0e884ed8836561665d8e0e7b: Skipping requested alter to schema version 2, tablet already version 2
I20260618 13:59:33.438557 29811 tablet.cc:1719] T ece7909d056c4ba8a5dca6423f0c1781 P 97859748af2c41879ba2a668dda999d2: Skipping requested alter to schema version 2, tablet already version 2
I20260618 13:59:33.442437 29303 consensus_queue.cc:1048] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:33.445025 29914 raft_consensus.cc:493] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.445231 29914 raft_consensus.cc:515] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.446504 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa9f65d8a5c46fdb745ee17af946dce" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:33.447121 28346 leader_election.cc:343] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.447978 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa9f65d8a5c46fdb745ee17af946dce" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:33.448750 28346 leader_election.cc:343] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.448881 28346 leader_election.cc:304] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:33.449178 29914 leader_election.cc:290] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:33.449509 29914 raft_consensus.cc:2749] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:33.453032 28797 tablet_bootstrap.cc:492] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:33.454075 28797 ts_tablet_manager.cc:1403] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.137s user 0.006s sys 0.000s
I20260618 13:59:33.455415 28797 raft_consensus.cc:359] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.456970 28797 raft_consensus.cc:385] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.457312 28797 raft_consensus.cc:740] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.457965 28798 tablet_bootstrap.cc:492] T 72fd63e8245d4042b091593881ba11b6 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:33.458130 28798 ts_tablet_manager.cc:1403] T 72fd63e8245d4042b091593881ba11b6 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.058s user 0.004s sys 0.004s
I20260618 13:59:33.458433 28797 consensus_queue.cc:260] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.460057 28798 raft_consensus.cc:359] T 72fd63e8245d4042b091593881ba11b6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.460270 28798 raft_consensus.cc:385] T 72fd63e8245d4042b091593881ba11b6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.460337 28798 raft_consensus.cc:740] T 72fd63e8245d4042b091593881ba11b6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.460600 28798 consensus_queue.cc:260] T 72fd63e8245d4042b091593881ba11b6 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.462500 28798 ts_tablet_manager.cc:1434] T 72fd63e8245d4042b091593881ba11b6 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 13:59:33.462913 28797 ts_tablet_manager.cc:1434] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260618 13:59:33.463989 28798 tablet_bootstrap.cc:492] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:33.465626 28797 tablet_bootstrap.cc:492] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:33.469959 28799 tablet_bootstrap.cc:492] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:33.470146 28799 ts_tablet_manager.cc:1403] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.080s user 0.010s sys 0.000s
I20260618 13:59:33.471074 28797 tablet_bootstrap.cc:654] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.471120 28799 raft_consensus.cc:359] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.471306 28799 raft_consensus.cc:385] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.471352 28799 raft_consensus.cc:740] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.471752 28799 consensus_queue.cc:260] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.472929 28799 ts_tablet_manager.cc:1434] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260618 13:59:33.473237 28799 tablet_bootstrap.cc:492] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:33.477173 29679 raft_consensus.cc:493] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.477337 28548 raft_consensus.cc:3060] T 39b06ae7444a48dbbec9e096fd7e4052 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.477344 29679 raft_consensus.cc:515] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.478253 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15483938a06e459789f87c6bb456bd15" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:33.478789 28346 leader_election.cc:343] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.479938 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15483938a06e459789f87c6bb456bd15" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:33.480372 28346 leader_election.cc:343] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.480470 28346 leader_election.cc:304] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:33.480659 29914 raft_consensus.cc:2749] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:33.480662 29679 leader_election.cc:290] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:33.495714 28652 tablet_service.cc:1511] Processing CreateTablet for tablet ba2d6894bf5649fc8cfae7e033c432b9 (DEFAULT_TABLE table=default.Table_dba55442642c45b1b8dfe959d62699c9 [id=596cac3f4f6540019b75bb586ec7f9ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.497031 28388 tablet_service.cc:1511] Processing CreateTablet for tablet ba2d6894bf5649fc8cfae7e033c432b9 (DEFAULT_TABLE table=default.Table_dba55442642c45b1b8dfe959d62699c9 [id=596cac3f4f6540019b75bb586ec7f9ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.498090 28659 tablet_service.cc:1511] Processing CreateTablet for tablet 08132883fb4f4632af2961bead244b18 (DEFAULT_TABLE table=default.Table_7c9d04c2f42d481e9ec6c499648cd688 [id=e5400836d2e64a5899f9549f3418cd52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.499370 28372 tablet_service.cc:1511] Processing CreateTablet for tablet 08132883fb4f4632af2961bead244b18 (DEFAULT_TABLE table=default.Table_7c9d04c2f42d481e9ec6c499648cd688 [id=e5400836d2e64a5899f9549f3418cd52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.503731 28663 tablet_service.cc:1511] Processing CreateTablet for tablet 779755a607bf46219900412b527e58b5 (DEFAULT_TABLE table=default.Table_ce0dbbee92c44d35b8deaf3d68303529 [id=15342e959fa343d8bf9faaba605a7d86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.507575 28375 tablet_service.cc:1511] Processing CreateTablet for tablet 779755a607bf46219900412b527e58b5 (DEFAULT_TABLE table=default.Table_ce0dbbee92c44d35b8deaf3d68303529 [id=15342e959fa343d8bf9faaba605a7d86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.508566 28656 tablet_service.cc:1511] Processing CreateTablet for tablet 5ffd607dd9eb43e0a3769fcdd28ed05f (DEFAULT_TABLE table=default.Table_e4b61938ff2f43ba97707a336462a018 [id=25e91888e13343c18166da32e455f504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.509955 28381 tablet_service.cc:1511] Processing CreateTablet for tablet 5ffd607dd9eb43e0a3769fcdd28ed05f (DEFAULT_TABLE table=default.Table_e4b61938ff2f43ba97707a336462a018 [id=25e91888e13343c18166da32e455f504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.510718 28657 tablet_service.cc:1511] Processing CreateTablet for tablet 732c8801c319400283806aac7f15917e (DEFAULT_TABLE table=default.Table_1cd2893d47c743138f1257093bd5b7b7 [id=4673506fe78d4bd99a54a0e4afac1501]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.510962 29714 consensus_queue.cc:1048] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:33.511104 28797 tablet_bootstrap.cc:492] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:33.511205 28797 ts_tablet_manager.cc:1403] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.047s user 0.006s sys 0.009s
I20260618 13:59:33.511806 28512 tablet_service.cc:1511] Processing CreateTablet for tablet ba2d6894bf5649fc8cfae7e033c432b9 (DEFAULT_TABLE table=default.Table_dba55442642c45b1b8dfe959d62699c9 [id=596cac3f4f6540019b75bb586ec7f9ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.511982 28514 tablet_service.cc:1511] Processing CreateTablet for tablet 08132883fb4f4632af2961bead244b18 (DEFAULT_TABLE table=default.Table_7c9d04c2f42d481e9ec6c499648cd688 [id=e5400836d2e64a5899f9549f3418cd52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.512181 28512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba2d6894bf5649fc8cfae7e033c432b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.512387 28514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08132883fb4f4632af2961bead244b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.513018 28797 raft_consensus.cc:359] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.513190 28797 raft_consensus.cc:385] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.513253 28797 raft_consensus.cc:740] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.511982 29759 raft_consensus.cc:493] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.513454 28797 consensus_queue.cc:260] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.515115 28797 ts_tablet_manager.cc:1434] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 13:59:33.515630 29745 ts_tablet_manager.cc:1929] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:33.515746 29745 log.cc:1199] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/316f470ea82f42fca48e4e96d49c9574
I20260618 13:59:33.513355 29759 raft_consensus.cc:515] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.516081 29745 ts_tablet_manager.cc:1950] T 316f470ea82f42fca48e4e96d49c9574 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:33.516448 29566 ts_tablet_manager.cc:1929] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:33.516536 29566 log.cc:1199] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/b6dd422c05324c989e29b20ae2afabad
I20260618 13:59:33.516844 29566 ts_tablet_manager.cc:1950] T b6dd422c05324c989e29b20ae2afabad P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:33.517041 28683 raft_consensus.cc:2468] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:33.517424 28408 raft_consensus.cc:2468] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:33.520012 28799 tablet_bootstrap.cc:654] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.520640 28652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba2d6894bf5649fc8cfae7e033c432b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.521122 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08132883fb4f4632af2961bead244b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.521517 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 779755a607bf46219900412b527e58b5 (DEFAULT_TABLE table=default.Table_ce0dbbee92c44d35b8deaf3d68303529 [id=15342e959fa343d8bf9faaba605a7d86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.521870 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 779755a607bf46219900412b527e58b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.522955 28797 tablet_bootstrap.cc:492] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:33.524507 28797 tablet_bootstrap.cc:654] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.526005 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 779755a607bf46219900412b527e58b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.526460 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffd607dd9eb43e0a3769fcdd28ed05f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.526927 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 732c8801c319400283806aac7f15917e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.531342 28388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba2d6894bf5649fc8cfae7e033c432b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.531781 28372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08132883fb4f4632af2961bead244b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.535352 28346 leader_election.cc:304] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:33.536502 28374 tablet_service.cc:1511] Processing CreateTablet for tablet 732c8801c319400283806aac7f15917e (DEFAULT_TABLE table=default.Table_1cd2893d47c743138f1257093bd5b7b7 [id=4673506fe78d4bd99a54a0e4afac1501]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.536772 29679 raft_consensus.cc:2804] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:33.536886 29679 raft_consensus.cc:697] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:33.537149 29679 consensus_queue.cc:237] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.538271 29914 raft_consensus.cc:493] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.538398 29914 raft_consensus.cc:515] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.538648 28617 leader_election.cc:304] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:33.539125 29914 leader_election.cc:290] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:33.539199 29755 raft_consensus.cc:2749] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:33.539389 29755 raft_consensus.cc:493] T 72fd63e8245d4042b091593881ba11b6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.539491 29755 raft_consensus.cc:515] T 72fd63e8245d4042b091593881ba11b6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.540828 28375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 779755a607bf46219900412b527e58b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.541127 29303 raft_consensus.cc:2804] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:33.541255 29303 raft_consensus.cc:697] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:33.541510 29303 consensus_queue.cc:237] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.545181 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bc87770baa34842a25ed97a19354446" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:33.545872 28346 leader_election.cc:343] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.546106 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bc87770baa34842a25ed97a19354446" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:33.547521 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 47) took 1523 ms. Trace:
I20260618 13:59:33.547600 29566 rpcz_store.cc:276] 0618 13:59:32.024267 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.024311 (+ 44us) service_pool.cc:224] Handling call
0618 13:59:33.352416 (+1328105us) ts_tablet_manager.cc:1151] Deleting tablet b6dd422c05324c989e29b20ae2afabad
0618 13:59:33.352426 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:33.427645 (+ 75219us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:33.516395 (+ 88750us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:33.547506 (+ 31111us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1327970}
I20260618 13:59:33.547940 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 48) took 1522 ms. Trace:
I20260618 13:59:33.548005 29566 rpcz_store.cc:276] 0618 13:59:32.025208 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.025253 (+ 45us) service_pool.cc:224] Handling call
0618 13:59:33.547847 (+1522594us) ts_tablet_manager.cc:1151] Deleting tablet 233040a16b0d4b61b977ca57c1afec2d
0618 13:59:33.547856 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:33.547924 (+ 68us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1522456}
I20260618 13:59:33.548331 29566 tablet_replica.cc:333] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:33.548331 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet b6dd422c05324c989e29b20ae2afabad (table default.Table_9424eec16a284672a0fe7ab3d0852b40 [id=070ea48af78346aa8ff43f2bb369730c]) successfully deleted
I20260618 13:59:33.548506 29566 raft_consensus.cc:2243] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:33.548732 29566 raft_consensus.cc:2272] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:33.548831 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 233040a16b0d4b61b977ca57c1afec2d because the tablet was not found. No further retry: Not found: Tablet not found: 233040a16b0d4b61b977ca57c1afec2d
I20260618 13:59:33.549883 29755 leader_election.cc:290] T 72fd63e8245d4042b091593881ba11b6 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:33.551398 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72fd63e8245d4042b091593881ba11b6" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:33.551640 28411 raft_consensus.cc:2393] T 72fd63e8245d4042b091593881ba11b6 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:33.552795 29759 leader_election.cc:290] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:33.553406 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72fd63e8245d4042b091593881ba11b6" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:33.553833 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f278d0acbb0c4a0ca61640bc142e6335" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:33.554312 28483 leader_election.cc:343] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: f278d0acbb0c4a0ca61640bc142e6335
W20260618 13:59:33.554955 28618 consensus_peers.cc:597] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:33.555186 28656 tablet_service.cc:1511] Processing CreateTablet for tablet dffeeb7313714c3f9eccc228c98ae7a3 (DEFAULT_TABLE table=default.Table_aab2bb1a1fc6434b9720650bdc13c115 [id=fa4f3da35d544fd99fd6c01264dd6fd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.555868 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dffeeb7313714c3f9eccc228c98ae7a3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:33.556208 28346 leader_election.cc:343] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.556330 28346 leader_election.cc:304] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:33.556943 28525 tablet_service.cc:1511] Processing CreateTablet for tablet 5ffd607dd9eb43e0a3769fcdd28ed05f (DEFAULT_TABLE table=default.Table_e4b61938ff2f43ba97707a336462a018 [id=25e91888e13343c18166da32e455f504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.558102 29914 raft_consensus.cc:2749] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:33.560256 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffd607dd9eb43e0a3769fcdd28ed05f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.560446 28484 leader_election.cc:304] T 72fd63e8245d4042b091593881ba11b6 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:33.560876 28484 leader_election.cc:304] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:33.561219 29755 raft_consensus.cc:3060] T 72fd63e8245d4042b091593881ba11b6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.561440 29759 raft_consensus.cc:2749] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:33.563098 28390 tablet_service.cc:1511] Processing CreateTablet for tablet dffeeb7313714c3f9eccc228c98ae7a3 (DEFAULT_TABLE table=default.Table_aab2bb1a1fc6434b9720650bdc13c115 [id=fa4f3da35d544fd99fd6c01264dd6fd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.563839 29566 ts_tablet_manager.cc:1916] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:33.564514 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:33.564797 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet ece7909d056c4ba8a5dca6423f0c1781 (table default.Table_f7e1e81727cb4aa8a4e65ddc115b49c0 [id=803d5220ce1a4dcfaf27d93c574be2ac]). Expected version 2 got 0
I20260618 13:59:33.564993 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:33.565285 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:33.565464 29025 catalog_manager.cc:5696] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } }
I20260618 13:59:33.565769 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 15180aefba0544329daee74308ef5010 (table default.Table_97b6bb39b3bf4612be391b9dd78d1646 [id=42a2a24590834abc9537669aaf223a7c]). Expected version 1 got 0
I20260618 13:59:33.566084 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:33.566217 29644 raft_consensus.cc:493] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.566335 29644 raft_consensus.cc:515] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.566606 28513 tablet_service.cc:1511] Processing CreateTablet for tablet dffeeb7313714c3f9eccc228c98ae7a3 (DEFAULT_TABLE table=default.Table_aab2bb1a1fc6434b9720650bdc13c115 [id=fa4f3da35d544fd99fd6c01264dd6fd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.566949 29644 leader_election.cc:290] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:33.567018 28513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dffeeb7313714c3f9eccc228c98ae7a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.567432 28663 tablet_service.cc:1511] Processing CreateTablet for tablet f32d411b71d146059d40260e54c283ec (DEFAULT_TABLE table=default.Table_82535d8562ad4142860544cc6cbf6fc2 [id=c630ac88fabc4db5a0e7d38fb686b09f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.568132 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f32d411b71d146059d40260e54c283ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.569480 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5a334d18ba4c06bb358fa86d8af80d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:33.569623 28411 raft_consensus.cc:2468] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:33.570173 28617 leader_election.cc:304] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:33.570405 29644 consensus_queue.cc:1048] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:33.570735 29644 raft_consensus.cc:2804] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:33.570803 29644 raft_consensus.cc:493] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:33.570871 29644 raft_consensus.cc:3060] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.570803 28515 tablet_service.cc:1511] Processing CreateTablet for tablet 732c8801c319400283806aac7f15917e (DEFAULT_TABLE table=default.Table_1cd2893d47c743138f1257093bd5b7b7 [id=4673506fe78d4bd99a54a0e4afac1501]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.571172 28515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 732c8801c319400283806aac7f15917e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.575328 28381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffd607dd9eb43e0a3769fcdd28ed05f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.579528 28663 tablet_service.cc:1511] Processing CreateTablet for tablet cb7f2d02ec004b9298b94307c7a38a97 (DEFAULT_TABLE table=default.Table_b2262e1c515848f7b545886563ac854a [id=e4288a5f828c4c0c91c9c174683d0a58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.579877 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb7f2d02ec004b9298b94307c7a38a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.580343 28539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5a334d18ba4c06bb358fa86d8af80d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:33.578545 29811 tablet.cc:1724] T dc36fbea853a4d4e9f61909200239aed P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
I20260618 13:59:33.581720 29775 tablet.cc:1724] T dc36fbea853a4d4e9f61909200239aed P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
I20260618 13:59:33.587077 28656 tablet_service.cc:1511] Processing CreateTablet for tablet e438b3733c374e1bb8aae4ed9f16b4a4 (DEFAULT_TABLE table=default.Table_39e2396806924281b1b78edd2295809b [id=57e78201a23a420f80f89ed2c6cbd3d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.587457 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e438b3733c374e1bb8aae4ed9f16b4a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.591048 28798 tablet_bootstrap.cc:654] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.591598 28510 tablet_service.cc:1511] Processing CreateTablet for tablet f32d411b71d146059d40260e54c283ec (DEFAULT_TABLE table=default.Table_82535d8562ad4142860544cc6cbf6fc2 [id=c630ac88fabc4db5a0e7d38fb686b09f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.591992 28510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f32d411b71d146059d40260e54c283ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.592978 28659 tablet_service.cc:1511] Processing CreateTablet for tablet c200f34abf074684a0314c9024f04caf (DEFAULT_TABLE table=default.Table_e57bb556500249678a1689b336e0335c [id=2beaf132552749f38a4b64a3c6fc4265]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.593288 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c200f34abf074684a0314c9024f04caf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.594038 29782 tablet.cc:1724] T dc36fbea853a4d4e9f61909200239aed P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
I20260618 13:59:33.595060 28384 tablet_service.cc:1511] Processing CreateTablet for tablet f32d411b71d146059d40260e54c283ec (DEFAULT_TABLE table=default.Table_82535d8562ad4142860544cc6cbf6fc2 [id=c630ac88fabc4db5a0e7d38fb686b09f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.595444 28386 tablet_service.cc:1511] Processing CreateTablet for tablet cb7f2d02ec004b9298b94307c7a38a97 (DEFAULT_TABLE table=default.Table_b2262e1c515848f7b545886563ac854a [id=e4288a5f828c4c0c91c9c174683d0a58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.595748 28377 tablet_service.cc:1511] Processing CreateTablet for tablet e438b3733c374e1bb8aae4ed9f16b4a4 (DEFAULT_TABLE table=default.Table_39e2396806924281b1b78edd2295809b [id=57e78201a23a420f80f89ed2c6cbd3d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.566375 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:33.597569 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:33.598598 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:33.600697 28383 tablet_service.cc:1511] Processing CreateTablet for tablet c200f34abf074684a0314c9024f04caf (DEFAULT_TABLE table=default.Table_e57bb556500249678a1689b336e0335c [id=2beaf132552749f38a4b64a3c6fc4265]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.601094 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:33.601529 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet dc36fbea853a4d4e9f61909200239aed (table default.Table_dbc6162c06f74e71b792bd1f7b2388fe [id=49576dabdecf41aca1b15ff3fc0c113c]). Expected version 6 got 0
W20260618 13:59:33.601732 28618 leader_election.cc:343] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.601822 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a]). Expected version 1 got 0
I20260618 13:59:33.602047 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 39b06ae7444a48dbbec9e096fd7e4052 (table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15]). Expected version 2 got 0
I20260618 13:59:33.602268 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:33.602504 29025 catalog_manager.cc:5696] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:33.605667 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet cea4bc13eabc4dd98b4f96723498014c (table default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c]). Expected version 1 got 0
I20260618 13:59:33.606213 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:33.606848 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:33.607563 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:33.608155 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_7f787ab851be496dae97cba8835db19b [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 1 got 0
I20260618 13:59:33.608682 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:33.609153 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:33.609699 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:33.610387 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:33.611057 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:33.607088 28528 tablet_service.cc:1511] Processing CreateTablet for tablet cb7f2d02ec004b9298b94307c7a38a97 (DEFAULT_TABLE table=default.Table_b2262e1c515848f7b545886563ac854a [id=e4288a5f828c4c0c91c9c174683d0a58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.611506 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:33.611889 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:33.612478 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_efb2b8e451084b7ba0de07e0a6e538c2 [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 1 got 0
I20260618 13:59:33.613124 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:33.613639 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:33.614125 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb7f2d02ec004b9298b94307c7a38a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.619403 29644 raft_consensus.cc:515] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.619717 29914 raft_consensus.cc:493] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.619867 29914 raft_consensus.cc:515] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.620131 29644 leader_election.cc:290] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:33.620446 29914 leader_election.cc:290] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:33.625129 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5a334d18ba4c06bb358fa86d8af80d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:33.625371 28403 raft_consensus.cc:3060] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.630539 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "650ed7ee8b134bc4b9f95028c54f99be" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:33.631351 28346 leader_election.cc:343] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.631659 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:33.634966 29719 raft_consensus.cc:493] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.635089 29719 raft_consensus.cc:515] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.635679 29719 leader_election.cc:290] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:33.635752 29714 consensus_queue.cc:1048] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:33.636368 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5a334d18ba4c06bb358fa86d8af80d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:33.637261 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04dac931e62a448d99633878eb70b92f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:33.637517 28411 raft_consensus.cc:2468] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:33.637893 28617 leader_election.cc:304] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:33.638255 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:33.638464 28799 tablet_bootstrap.cc:492] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:33.638523 29714 raft_consensus.cc:2804] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:33.638593 28799 ts_tablet_manager.cc:1403] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.165s user 0.015s sys 0.038s
I20260618 13:59:33.638612 29714 raft_consensus.cc:493] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:33.638674 29714 raft_consensus.cc:3060] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.639053 29025 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet f45556a59c424ca2bc30956566374c89 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:33.639164 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "650ed7ee8b134bc4b9f95028c54f99be" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:33.639303 28522 tablet_service.cc:1511] Processing CreateTablet for tablet c200f34abf074684a0314c9024f04caf (DEFAULT_TABLE table=default.Table_e57bb556500249678a1689b336e0335c [id=2beaf132552749f38a4b64a3c6fc4265]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:33.639855 28346 leader_election.cc:343] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.639921 28799 raft_consensus.cc:359] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.640110 28799 raft_consensus.cc:385] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.640192 28799 raft_consensus.cc:740] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.640027 28346 leader_election.cc:304] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:33.640390 28524 tablet_service.cc:1511] Processing CreateTablet for tablet e438b3733c374e1bb8aae4ed9f16b4a4 (DEFAULT_TABLE table=default.Table_39e2396806924281b1b78edd2295809b [id=57e78201a23a420f80f89ed2c6cbd3d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:33.640554 29011 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 46bfe8cc2078414d84e65e22ed4ef8a2
I20260618 13:59:33.640583 29914 raft_consensus.cc:2749] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:33.640900 28799 consensus_queue.cc:260] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.641129 29011 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 63) took 1522 ms. Trace:
I20260618 13:59:33.641330 28373 tablet_service.cc:1558] Processing DeleteTablet for tablet 46bfe8cc2078414d84e65e22ed4ef8a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:33.641269 29011 rpcz_store.cc:276] 0618 13:59:32.119029 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.875327 (+756298us) service_pool.cc:224] Handling call
0618 13:59:32.875794 (+ 467us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:32.875936 (+ 142us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:32.875996 (+ 60us) catalog_manager.cc:2849] Locking tablets
0618 13:59:33.174637 (+298641us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:33.270693 (+ 96056us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:33.270773 (+ 80us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:33.640451 (+369678us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:33.641056 (+ 605us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:33.180740 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:33.180846 (+ 106us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:33.180852 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:33.180854 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:33.180965 (+ 111us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:33.180974 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:33.180980 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:33.181116 (+ 136us) tablet.cc:701] Row locks acquired
0618 13:59:33.181118 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:33.181151 (+ 33us) write_op.cc:270] Start()
0618 13:59:33.181180 (+ 29us) write_op.cc:276] Timestamp: P: 1781791173180742 usec, L: 0
0618 13:59:33.181183 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:33.208177 (+ 26994us) log.cc:844] Serialized 762 byte log entry
0618 13:59:33.208377 (+ 200us) spinlock_profiling.cc:243] Waited 26.8 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fcbac75ef22 00007fcbadb40305 00007fcbadb3ee29 00007fcbadb47250 00007fcbadb4705d 0000000000aa681c 00007fcba07299ff 00007fcba072cd1b 00007fcba072cb98 0000000000aa681c
0618 13:59:33.270178 (+ 61801us) op_driver.cc:464] REPLICATION: finished
0618 13:59:33.270249 (+ 71us) write_op.cc:301] APPLY: starting
0618 13:59:33.270262 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:33.270296 (+ 34us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:33.270299 (+ 3us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:33.270368 (+ 69us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:33.270395 (+ 27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:33.270401 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:33.270477 (+ 76us) log.cc:844] Serialized 29 byte log entry
0618 13:59:33.270547 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:33.270561 (+ 14us) write_op.cc:454] Released schema lock
0618 13:59:33.270568 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":30,"apply.run_cpu_time_us":374,"apply.run_wall_time_us":374,"num_ops":2,"prepare.queue_time_us":5463,"prepare.run_cpu_time_us":1377,"prepare.run_wall_time_us":27708,"raft.queue_time_us":1658,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":249,"replication_time_us":88970,"spinlock_wait_cycles":61611264}]]}
I20260618 13:59:33.642027 28799 ts_tablet_manager.cc:1434] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:33.641983 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04dac931e62a448d99633878eb70b92f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:33.650761 28618 leader_election.cc:343] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:33.651648 28618 leader_election.cc:343] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.653462 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 46bfe8cc2078414d84e65e22ed4ef8a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:33.657728 29644 raft_consensus.cc:493] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.657851 29644 raft_consensus.cc:515] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.658601 29644 leader_election.cc:290] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:33.658778 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1baf86497d442dbbbddbc3498bebbee" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:33.659020 28411 raft_consensus.cc:2468] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:33.659538 28617 leader_election.cc:304] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:33.659825 29644 raft_consensus.cc:2804] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:33.659906 29644 raft_consensus.cc:493] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:33.659968 29644 raft_consensus.cc:3060] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.661036 28797 tablet_bootstrap.cc:492] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:33.661165 28797 ts_tablet_manager.cc:1403] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.138s user 0.006s sys 0.010s
I20260618 13:59:33.661852 28797 raft_consensus.cc:359] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.662005 28797 raft_consensus.cc:385] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.662063 28797 raft_consensus.cc:740] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.662285 28797 consensus_queue.cc:260] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.662603 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 46bfe8cc2078414d84e65e22ed4ef8a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:33.662753 28797 ts_tablet_manager.cc:1434] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:33.664696 28797 tablet_bootstrap.cc:492] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:33.665441 29746 tablet_replica.cc:333] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:33.665688 29746 raft_consensus.cc:2243] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:33.665880 29746 raft_consensus.cc:2272] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:33.666322 28797 tablet_bootstrap.cc:654] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.666577 29746 ts_tablet_manager.cc:1916] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:33.678654 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1baf86497d442dbbbddbc3498bebbee" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:33.679612 28618 leader_election.cc:343] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.695314 28797 tablet_bootstrap.cc:492] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:33.695468 28797 ts_tablet_manager.cc:1403] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.031s user 0.004s sys 0.000s
I20260618 13:59:33.697571 28797 raft_consensus.cc:359] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.697757 28797 raft_consensus.cc:385] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.703404 28544 raft_consensus.cc:3060] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.710350 28374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 732c8801c319400283806aac7f15917e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.710800 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f32d411b71d146059d40260e54c283ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.714779 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb7f2d02ec004b9298b94307c7a38a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.719174 28522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c200f34abf074684a0314c9024f04caf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.719553 29714 raft_consensus.cc:515] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.720170 29714 leader_election.cc:290] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:33.720482 28377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e438b3733c374e1bb8aae4ed9f16b4a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.721235 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e438b3733c374e1bb8aae4ed9f16b4a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.727509 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04dac931e62a448d99633878eb70b92f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:33.727778 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c200f34abf074684a0314c9024f04caf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:33.728196 28618 leader_election.cc:343] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.731483 28799 tablet_bootstrap.cc:492] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:33.731854 29745 tablet_replica.cc:333] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:33.732045 29745 raft_consensus.cc:2243] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:33.732158 28390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dffeeb7313714c3f9eccc228c98ae7a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:33.732204 29745 raft_consensus.cc:2272] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:33.732755 29745 ts_tablet_manager.cc:1916] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:33.733196 28799 tablet_bootstrap.cc:654] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.734179 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:33.734576 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:33.736900 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04dac931e62a448d99633878eb70b92f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:33.737073 28410 raft_consensus.cc:3060] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:33.739025 28618 consensus_peers.cc:597] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:33.739506 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:33.739857 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:33.740074 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:33.735637 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 316f470ea82f42fca48e4e96d49c9574 (table default.Table_28c746533f45405791fc12912e8fbcd4 [id=5415980263ed4fbea10b969e3b8facb4]) successfully deleted
I20260618 13:59:33.741487 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:33.741719 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:33.742143 29755 raft_consensus.cc:2749] T 72fd63e8245d4042b091593881ba11b6 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:33.742046 29011 catalog_manager.cc:5696] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:33.741132 28797 raft_consensus.cc:740] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.743346 28797 consensus_queue.cc:260] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.743654 29011 catalog_manager.cc:5696] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:33.745946 28797 ts_tablet_manager.cc:1434] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.050s user 0.018s sys 0.000s
I20260618 13:59:33.746363 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:33.747637 28797 tablet_bootstrap.cc:492] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:33.747963 28799 tablet_bootstrap.cc:492] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:33.748093 28799 ts_tablet_manager.cc:1403] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.005s
I20260618 13:59:33.748728 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:33.748848 28799 raft_consensus.cc:359] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.749015 28799 raft_consensus.cc:385] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.749073 28799 raft_consensus.cc:740] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.749310 28799 consensus_queue.cc:260] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.749902 28799 ts_tablet_manager.cc:1434] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:33.750144 28799 tablet_bootstrap.cc:492] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:33.750933 28797 tablet_bootstrap.cc:654] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.752102 28799 tablet_bootstrap.cc:654] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.752802 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:33.753394 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:33.754055 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet f45556a59c424ca2bc30956566374c89 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:33.754433 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_7f787ab851be496dae97cba8835db19b [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 1 got 0
I20260618 13:59:33.754824 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:33.755560 29400 raft_consensus.cc:493] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.755682 29400 raft_consensus.cc:515] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.756233 28799 tablet_bootstrap.cc:492] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:33.756270 29400 leader_election.cc:290] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:33.756350 28799 ts_tablet_manager.cc:1403] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260618 13:59:33.757064 28799 raft_consensus.cc:359] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.757233 28799 raft_consensus.cc:385] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.757298 28799 raft_consensus.cc:740] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.758695 28799 consensus_queue.cc:260] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.759615 28799 ts_tablet_manager.cc:1434] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260618 13:59:33.759867 28799 tablet_bootstrap.cc:492] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:33.760476 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c75278db2234b11bf95f1cf5cc76edd" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:33.760684 28409 raft_consensus.cc:2468] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:33.760897 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c75278db2234b11bf95f1cf5cc76edd" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:33.761202 28617 leader_election.cc:304] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:33.761492 28799 tablet_bootstrap.cc:654] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:33.761693 28618 leader_election.cc:343] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.762041 29400 raft_consensus.cc:2804] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:33.762112 29400 raft_consensus.cc:493] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:33.762166 29400 raft_consensus.cc:3060] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.765455 28505 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:34078 (request call id 118) took 1068 ms. Trace:
I20260618 13:59:33.765560 28505 rpcz_store.cc:276] 0618 13:59:32.697166 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.697243 (+ 77us) service_pool.cc:224] Handling call
0618 13:59:32.945756 (+248513us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x6160000a5754. stack: 00007fc19811e496 00007fc19811e250 00007fc1971493ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fc1a41938c0 00007fc1a8044f1f 00007fc1a2466c36 00007fc1a2466965 00007fc19f22888e 00007fc19f22776e 00007fc19f22d9a8 00007fc19f22e30f 00007fc19f22e138 0000000000aa681c
0618 13:59:33.322108 (+376352us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x6160000f6a54. stack: 00007fc19811e496 00007fc19811e250 00007fc1971493ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fc1a41938c0 00007fc1a8044f1f 00007fc1a2466c36 00007fc1a2466965 00007fc19f22888e 00007fc19f22776e 00007fc19f22d9a8 00007fc19f22e30f 00007fc19f22e138 0000000000aa681c
0618 13:59:33.330367 (+ 8259us) spinlock_profiling.cc:243] Waited 7.15 ms on lock 0x616000123454. stack: 00007fc19811e496 00007fc19811e250 00007fc1971493ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fc1a41938c0 00007fc1a8044f1f 00007fc1a2466c36 00007fc1a2466965 00007fc19f22888e 00007fc19f22776e 00007fc19f22d9a8 00007fc19f22e30f 00007fc19f22e138 0000000000aa681c
0618 13:59:33.550538 (+220171us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x616000174454. stack: 00007fc19811e496 00007fc19811e250 00007fc1971493ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fc1a41938c0 00007fc1a8044f1f 00007fc1a2466c36 00007fc1a2466965 00007fc19f22888e 00007fc19f22776e 00007fc19f22d9a8 00007fc19f22e30f 00007fc19f22e138 0000000000aa681c
0618 13:59:33.618617 (+ 68079us) spinlock_profiling.cc:243] Waited 67.6 ms on lock 0x6160000f7054. stack: 00007fc19811e496 00007fc19811e250 00007fc1971493ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fc1a41938c0 00007fc1a8044f1f 00007fc1a2466c36 00007fc1a2466965 00007fc19f22888e 00007fc19f22776e 00007fc19f22d9a8 00007fc19f22e30f 00007fc19f22e138 0000000000aa681c
0618 13:59:33.742444 (+123827us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x6160000c7054. stack: 00007fc19811e496 00007fc19811e250 00007fc1971493ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fc1a41938c0 00007fc1a8044f1f 00007fc1a2466c36 00007fc1a2466965 00007fc19f22888e 00007fc19f22776e 00007fc19f22d9a8 00007fc19f22e30f 00007fc19f22e138 0000000000aa681c
0618 13:59:33.765423 (+ 22979us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2353925760}
I20260618 13:59:33.767020 28403 raft_consensus.cc:2468] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:33.767446 28617 leader_election.cc:304] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:33.767684 29719 raft_consensus.cc:2804] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:33.767792 29719 raft_consensus.cc:697] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:33.768052 29719 consensus_queue.cc:237] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.780046 28797 tablet_bootstrap.cc:492] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:33.780192 28797 ts_tablet_manager.cc:1403] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20260618 13:59:33.780864 28797 raft_consensus.cc:359] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.781039 28797 raft_consensus.cc:385] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.781108 28797 raft_consensus.cc:740] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.781340 28797 consensus_queue.cc:260] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.781842 28797 ts_tablet_manager.cc:1434] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:33.782126 28797 tablet_bootstrap.cc:492] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:33.783605 28797 tablet_bootstrap.cc:654] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.783941 28798 tablet_bootstrap.cc:492] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:33.784241 28798 ts_tablet_manager.cc:1403] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.321s user 0.010s sys 0.003s
I20260618 13:59:33.785138 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:33.785223 28798 raft_consensus.cc:359] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.785557 28798 raft_consensus.cc:385] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.785674 28798 raft_consensus.cc:740] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.786010 28798 consensus_queue.cc:260] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.804098 28797 tablet_bootstrap.cc:492] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:33.804303 28797 ts_tablet_manager.cc:1403] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.005s
I20260618 13:59:33.805146 28797 raft_consensus.cc:359] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.805322 28797 raft_consensus.cc:385] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.805387 28797 raft_consensus.cc:740] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.805586 28797 consensus_queue.cc:260] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.806052 28797 ts_tablet_manager.cc:1434] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:33.806300 28798 ts_tablet_manager.cc:1434] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.022s user 0.000s sys 0.002s
I20260618 13:59:33.806316 28797 tablet_bootstrap.cc:492] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:33.808272 28797 tablet_bootstrap.cc:654] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.812750 29644 raft_consensus.cc:515] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.813331 29644 leader_election.cc:290] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:33.813613 28410 raft_consensus.cc:2468] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:33.816370 29566 ts_tablet_manager.cc:1929] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:33.816479 29566 log.cc:1199] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/aef7c1f111634c98af046e73fa9ffdeb
I20260618 13:59:33.816845 29566 ts_tablet_manager.cc:1950] T aef7c1f111634c98af046e73fa9ffdeb P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:33.823724 28617 leader_election.cc:304] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:33.824154 29644 raft_consensus.cc:2804] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:33.824262 29644 raft_consensus.cc:697] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:33.824479 29644 consensus_queue.cc:237] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.839507 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1baf86497d442dbbbddbc3498bebbee" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:33.839689 28410 raft_consensus.cc:3060] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.838068 29759 raft_consensus.cc:493] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:33.844040 29759 raft_consensus.cc:515] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.844965 29759 leader_election.cc:290] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:33.845577 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02b5cc08ca0c4e719e0414b345211ba5" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:33.853456 29719 consensus_queue.cc:1048] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:33.854995 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02b5cc08ca0c4e719e0414b345211ba5" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:33.855599 28484 leader_election.cc:304] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:33.856304 29755 raft_consensus.cc:2749] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:33.860491 28798 tablet_bootstrap.cc:492] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:33.868398 29640 consensus_queue.cc:1048] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:33.871294 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1baf86497d442dbbbddbc3498bebbee" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:33.871780 28618 leader_election.cc:343] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.878530 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:33.878803 29759 raft_consensus.cc:493] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.878914 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 15180aefba0544329daee74308ef5010 (table default.Table_97b6bb39b3bf4612be391b9dd78d1646 [id=42a2a24590834abc9537669aaf223a7c]). Expected version 1 got 0
I20260618 13:59:33.878981 29759 raft_consensus.cc:515] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.879189 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_efb2b8e451084b7ba0de07e0a6e538c2 [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 1 got 0
I20260618 13:59:33.879611 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:33.879710 29759 leader_election.cc:290] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:33.879935 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:33.880192 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet dc36fbea853a4d4e9f61909200239aed (table default.Table_dbc6162c06f74e71b792bd1f7b2388fe [id=49576dabdecf41aca1b15ff3fc0c113c]). Expected version 6 got 0
I20260618 13:59:33.880367 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:33.880578 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:33.880918 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 72c04f55890648f0a8184f3783243f7c (table default.Table_8783a8d8c526428281436b62b2581a33 [id=0d83fc68255c458587390c58b12e00d7]). Expected version 1 got 0
I20260618 13:59:33.880973 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aef7c1f111634c98af046e73fa9ffdeb" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:33.880995 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aef7c1f111634c98af046e73fa9ffdeb" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:33.881095 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet ece7909d056c4ba8a5dca6423f0c1781 (table default.Table_f7e1e81727cb4aa8a4e65ddc115b49c0 [id=803d5220ce1a4dcfaf27d93c574be2ac]). Expected version 2 got 0
I20260618 13:59:33.881747 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
W20260618 13:59:33.881909 28484 leader_election.cc:343] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: Tablet not RUNNING: STOPPED
I20260618 13:59:33.882030 28484 leader_election.cc:304] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:33.882441 29759 raft_consensus.cc:2749] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:33.882171 29011 catalog_manager.cc:5696] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:33.886554 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 39b06ae7444a48dbbec9e096fd7e4052 (table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15]). Expected version 2 got 0
I20260618 13:59:33.887396 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a]). Expected version 1 got 0
I20260618 13:59:33.888366 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:33.889011 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet cea4bc13eabc4dd98b4f96723498014c (table default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c]). Expected version 1 got 0
W20260618 13:59:33.889430 28617 consensus_peers.cc:597] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d3dbf13544ce408ab97834aec5862390. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:33.889495 28797 tablet_bootstrap.cc:492] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:33.889693 28797 ts_tablet_manager.cc:1403] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.083s user 0.004s sys 0.005s
I20260618 13:59:33.889484 29011 catalog_manager.cc:5696] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:33.890586 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:33.891270 29011 catalog_manager.cc:5696] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:33.890601 28797 raft_consensus.cc:359] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.891889 28797 raft_consensus.cc:385] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.891963 28797 raft_consensus.cc:740] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.892226 28797 consensus_queue.cc:260] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.892812 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:33.893136 28797 ts_tablet_manager.cc:1434] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:33.893455 28797 tablet_bootstrap.cc:492] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:33.894630 29011 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:33.895694 28797 tablet_bootstrap.cc:654] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.896066 29914 raft_consensus.cc:493] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:33.896207 29914 raft_consensus.cc:515] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.896756 29914 leader_election.cc:290] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:33.897341 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f7359c4a144ca89fe3e51d1365a425" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:33.897746 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f7359c4a144ca89fe3e51d1365a425" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:33.898341 28346 leader_election.cc:343] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:33.899207 28346 leader_election.cc:343] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:33.899298 28346 leader_election.cc:304] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:33.899529 28798 tablet_bootstrap.cc:654] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.899531 29914 raft_consensus.cc:2749] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:33.901038 29017 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 9a0691de253b4b008e5d995fd6b876c2
I20260618 13:59:33.901557 29017 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 64) took 1779 ms. Trace:
I20260618 13:59:33.901640 29017 rpcz_store.cc:276] 0618 13:59:32.121774 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.877790 (+756016us) service_pool.cc:224] Handling call
0618 13:59:32.878163 (+ 373us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:32.878297 (+ 134us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:32.878347 (+ 50us) catalog_manager.cc:2849] Locking tablets
0618 13:59:33.175473 (+297126us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:33.271850 (+ 96377us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:33.271916 (+ 66us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:33.900984 (+629068us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:33.901507 (+ 523us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:33.208483 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:33.208609 (+ 126us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:33.208615 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:33.208617 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:33.208746 (+ 129us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:33.208757 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:33.208762 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:33.208874 (+ 112us) tablet.cc:701] Row locks acquired
0618 13:59:33.208876 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:33.208909 (+ 33us) write_op.cc:270] Start()
0618 13:59:33.208940 (+ 31us) write_op.cc:276] Timestamp: P: 1781791173208890 usec, L: 0
0618 13:59:33.208943 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:33.209067 (+ 124us) log.cc:844] Serialized 762 byte log entry
0618 13:59:33.271141 (+ 62074us) op_driver.cc:464] REPLICATION: finished
0618 13:59:33.271347 (+ 206us) write_op.cc:301] APPLY: starting
0618 13:59:33.271361 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:33.271394 (+ 33us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:33.271413 (+ 19us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:33.271471 (+ 58us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:33.271496 (+ 25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:33.271502 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:33.271554 (+ 52us) log.cc:844] Serialized 29 byte log entry
0618 13:59:33.271758 (+ 204us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:33.271775 (+ 17us) write_op.cc:454] Released schema lock
0618 13:59:33.271783 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:33.271967 (+ 184us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:33.271972 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":161,"apply.run_cpu_time_us":350,"apply.run_wall_time_us":614,"num_ops":2,"prepare.queue_time_us":32445,"prepare.run_cpu_time_us":7472,"prepare.run_wall_time_us":45888,"raft.queue_time_us":589,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"replication_time_us":62173,"thread_start_us":45170,"threads_started":1}]]}
I20260618 13:59:33.902469 29017 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_8783a8d8c526428281436b62b2581a33" } new_table_name: "default.Table_773be314d7e741fcb4fb6eb7d954802e" modify_external_catalogs: true
I20260618 13:59:33.905651 28799 tablet_bootstrap.cc:492] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:33.905779 28799 ts_tablet_manager.cc:1403] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.146s user 0.010s sys 0.054s
I20260618 13:59:33.906555 28799 raft_consensus.cc:359] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.906747 28799 raft_consensus.cc:385] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.906826 28799 raft_consensus.cc:740] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.907121 28799 consensus_queue.cc:260] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.907739 28799 ts_tablet_manager.cc:1434] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:33.908018 28799 tablet_bootstrap.cc:492] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:33.909641 28799 tablet_bootstrap.cc:654] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.909818 29400 raft_consensus.cc:515] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.910539 29400 leader_election.cc:290] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:33.910902 29746 ts_tablet_manager.cc:1929] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:33.911010 29746 log.cc:1199] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/46bfe8cc2078414d84e65e22ed4ef8a2
I20260618 13:59:33.911384 29746 ts_tablet_manager.cc:1950] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:33.911695 28410 raft_consensus.cc:2468] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:33.912654 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 49) took 1886 ms. Trace:
I20260618 13:59:33.912742 29566 rpcz_store.cc:276] 0618 13:59:32.025910 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.025967 (+ 57us) service_pool.cc:224] Handling call
0618 13:59:33.548269 (+1522302us) ts_tablet_manager.cc:1151] Deleting tablet aef7c1f111634c98af046e73fa9ffdeb
0618 13:59:33.548279 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:33.774871 (+226592us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:33.816314 (+ 41443us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:33.912632 (+ 96318us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1522070}
I20260618 13:59:33.913089 29566 tablet_replica.cc:333] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:33.913324 29566 raft_consensus.cc:2243] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:33.913619 29566 raft_consensus.cc:2272] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:33.914223 29566 ts_tablet_manager.cc:1916] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:33.918315 28617 leader_election.cc:304] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:33.918852 29952 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.100.60:42083 every 8 connection(s)
I20260618 13:59:33.918852 29476 rpc_server.cc:307] RPC server started. Bound to: 127.27.100.60:42083
I20260618 13:59:33.919250 28410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c75278db2234b11bf95f1cf5cc76edd" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:33.919471 28410 raft_consensus.cc:3060] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:33.921469 29476 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data/info.pb
I20260618 13:59:33.922364 29719 raft_consensus.cc:2804] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:33.922502 29719 raft_consensus.cc:697] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:33.922736 29719 consensus_queue.cc:237] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.923398 28799 tablet_bootstrap.cc:492] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:33.923571 28799 ts_tablet_manager.cc:1403] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.006s
I20260618 13:59:33.924273 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet aef7c1f111634c98af046e73fa9ffdeb (table default.Table_191169025bff42cab93cf42503fa7220 [id=9c29cca38acc493aaec62cf3b0d93fcf]) successfully deleted
I20260618 13:59:33.924471 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a0691de253b4b008e5d995fd6b876c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:33.924552 28799 raft_consensus.cc:359] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.924732 28799 raft_consensus.cc:385] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.924819 28799 raft_consensus.cc:740] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.924808 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a0691de253b4b008e5d995fd6b876c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:33.924906 28048 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu as pid 29476
I20260618 13:59:33.925033 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a0691de253b4b008e5d995fd6b876c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:33.925040 28799 consensus_queue.cc:260] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.925678 29746 tablet_replica.cc:333] T 9a0691de253b4b008e5d995fd6b876c2 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:33.925719 28799 ts_tablet_manager.cc:1434] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260618 13:59:33.925907 29746 raft_consensus.cc:2243] T 9a0691de253b4b008e5d995fd6b876c2 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:33.926018 28799 tablet_bootstrap.cc:492] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:33.926079 29746 raft_consensus.cc:2272] T 9a0691de253b4b008e5d995fd6b876c2 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:33.926764 29746 ts_tablet_manager.cc:1916] T 9a0691de253b4b008e5d995fd6b876c2 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:33.930354 28799 tablet_bootstrap.cc:654] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.930385 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 46bfe8cc2078414d84e65e22ed4ef8a2 (table default.Table_659a3d495ed44be4bfb2193816362e25 [id=2659803e822d4761a15e0e89644cc472]) successfully deleted
I20260618 13:59:33.932144 29745 ts_tablet_manager.cc:1929] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:33.932284 29745 log.cc:1199] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/fddea8fba4da40d7a5780e6750caf5e9
I20260618 13:59:33.932660 29745 ts_tablet_manager.cc:1950] T fddea8fba4da40d7a5780e6750caf5e9 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
W20260618 13:59:33.938287 28617 consensus_peers.cc:597] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:33.940606 28799 tablet_bootstrap.cc:492] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:33.940793 28799 ts_tablet_manager.cc:1403] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20260618 13:59:33.941574 28799 raft_consensus.cc:359] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.941787 28799 raft_consensus.cc:385] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.941862 28799 raft_consensus.cc:740] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.942493 28799 consensus_queue.cc:260] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.943200 28799 ts_tablet_manager.cc:1434] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260618 13:59:33.943508 28799 tablet_bootstrap.cc:492] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:33.945483 28799 tablet_bootstrap.cc:654] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:33.949658 29017 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 72c04f55890648f0a8184f3783243f7c consensus state has no leader
I20260618 13:59:33.951642 29963 sys_catalog.cc:263] Verifying existing consensus state
I20260618 13:59:33.957515 28799 tablet_bootstrap.cc:492] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:33.957705 28799 ts_tablet_manager.cc:1403] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.003s
I20260618 13:59:33.958655 28799 raft_consensus.cc:359] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.958868 28799 raft_consensus.cc:385] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.958973 28799 raft_consensus.cc:740] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.959266 28799 consensus_queue.cc:260] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.959569 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c75278db2234b11bf95f1cf5cc76edd" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:33.959925 28799 ts_tablet_manager.cc:1434] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260618 13:59:33.960320 28799 tablet_bootstrap.cc:492] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:33.960392 28618 leader_election.cc:343] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:33.960934 29019 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9ead386562eb4fd787c7aee776c2194f consensus state has no leader
I20260618 13:59:33.962572 28799 tablet_bootstrap.cc:654] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.967664 29963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Bootstrap starting.
I20260618 13:59:33.969995 29025 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_f958c4ce58c34bfda98454664e202af2 [id=7b747ef3b2494ca78c499c5bb75648d4] alter complete (version 2)
I20260618 13:59:33.971187 28799 tablet_bootstrap.cc:492] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:33.971437 28798 tablet_bootstrap.cc:492] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:33.971436 28799 ts_tablet_manager.cc:1403] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.004s
I20260618 13:59:33.971680 28798 ts_tablet_manager.cc:1403] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.111s user 0.002s sys 0.003s
I20260618 13:59:33.972294 28799 raft_consensus.cc:359] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.972510 28799 raft_consensus.cc:385] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.972609 28799 raft_consensus.cc:740] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.972877 28799 consensus_queue.cc:260] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.973532 29007 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 98b42d23832545ae81a7cdff19e4689f
I20260618 13:59:33.973809 28799 ts_tablet_manager.cc:1434] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:33.974164 28799 tablet_bootstrap.cc:492] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:33.974366 29007 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_0fbace1c1aa746a6a2a57c878631218b" } modify_external_catalogs: true
I20260618 13:59:33.976164 28799 tablet_bootstrap.cc:654] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:33.979493 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet 98b42d23832545ae81a7cdff19e4689f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:33 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:33.980187 28798 raft_consensus.cc:359] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.980455 28798 raft_consensus.cc:385] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.980563 28798 raft_consensus.cc:740] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.980881 28798 consensus_queue.cc:260] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:33.981577 28798 ts_tablet_manager.cc:1434] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260618 13:59:33.981906 28798 tablet_bootstrap.cc:492] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:33.983059 28797 tablet_bootstrap.cc:492] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:33.983210 28797 ts_tablet_manager.cc:1403] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.090s user 0.003s sys 0.000s
I20260618 13:59:33.984090 28797 raft_consensus.cc:359] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.984309 28797 raft_consensus.cc:385] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.984368 28797 raft_consensus.cc:740] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.984609 28797 consensus_queue.cc:260] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:33.985175 28797 ts_tablet_manager.cc:1434] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:33.985497 28797 tablet_bootstrap.cc:492] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:33.986115 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 98b42d23832545ae81a7cdff19e4689f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:33 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:33.986438 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 98b42d23832545ae81a7cdff19e4689f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:33 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:33.987195 28797 tablet_bootstrap.cc:654] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:33.990597 28618 consensus_peers.cc:597] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:33.993810 28799 tablet_bootstrap.cc:492] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:33.993952 28799 ts_tablet_manager.cc:1403] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20260618 13:59:33.993935 29714 consensus_queue.cc:1048] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:33.994776 28799 raft_consensus.cc:359] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.994952 28799 raft_consensus.cc:385] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:33.995066 28799 raft_consensus.cc:740] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:33.995324 28799 consensus_queue.cc:260] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:33.996106 28799 ts_tablet_manager.cc:1434] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260618 13:59:33.996439 28799 tablet_bootstrap.cc:492] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:33.998328 28799 tablet_bootstrap.cc:654] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.000089 28797 tablet_bootstrap.cc:492] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:34.000208 28797 ts_tablet_manager.cc:1403] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20260618 13:59:34.000852 28797 raft_consensus.cc:359] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.001006 28797 raft_consensus.cc:385] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.001053 28797 raft_consensus.cc:740] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.001248 28797 consensus_queue.cc:260] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.001507 29025 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_031e1a03030b45c99c1270d612008f91 [id=387d3ff1762749efad926181fe4c746a] alter complete (version 1)
I20260618 13:59:34.001762 28797 ts_tablet_manager.cc:1434] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260618 13:59:34.002017 28797 tablet_bootstrap.cc:492] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:34.003914 28797 tablet_bootstrap.cc:654] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.007582 29914 raft_consensus.cc:493] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.007833 29914 raft_consensus.cc:515] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.008883 29914 leader_election.cc:290] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.010432 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ead386562eb4fd787c7aee776c2194f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:34.011000 28346 leader_election.cc:343] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.011188 28799 tablet_bootstrap.cc:492] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:34.011389 28799 ts_tablet_manager.cc:1403] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20260618 13:59:34.012355 28799 raft_consensus.cc:359] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.012554 28799 raft_consensus.cc:385] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.012629 28799 raft_consensus.cc:740] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.012918 28799 consensus_queue.cc:260] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.013542 28799 ts_tablet_manager.cc:1434] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260618 13:59:34.013864 28799 tablet_bootstrap.cc:492] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:34.014066 29679 raft_consensus.cc:493] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.014195 29679 raft_consensus.cc:515] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.014794 29679 leader_election.cc:290] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.015858 28799 tablet_bootstrap.cc:654] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.017930 28797 tablet_bootstrap.cc:492] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:34.018074 28797 ts_tablet_manager.cc:1403] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.000s
I20260618 13:59:34.018751 29566 ts_tablet_manager.cc:1929] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:34.018846 29566 log.cc:1199] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/316f470ea82f42fca48e4e96d49c9574
I20260618 13:59:34.018955 28797 raft_consensus.cc:359] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.019122 28797 raft_consensus.cc:385] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.019181 28797 raft_consensus.cc:740] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.019412 28410 raft_consensus.cc:2468] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:34.019414 28797 consensus_queue.cc:260] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.019873 28797 ts_tablet_manager.cc:1434] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:34.020121 28797 tablet_bootstrap.cc:492] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:34.021688 28797 tablet_bootstrap.cc:654] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.022523 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30f4bd3f86484caab48c1f0806299460" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:34.023167 28346 leader_election.cc:343] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.023264 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 62) took 1116 ms. Trace:
I20260618 13:59:34.023490 28799 tablet_bootstrap.cc:492] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:34.023352 29745 rpcz_store.cc:276] 0618 13:59:32.906399 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.906509 (+ 110us) service_pool.cc:224] Handling call
0618 13:59:33.731813 (+825304us) ts_tablet_manager.cc:1151] Deleting tablet fddea8fba4da40d7a5780e6750caf5e9
0618 13:59:33.731821 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:33.812140 (+ 80319us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:33.932081 (+119941us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:34.023196 (+ 91115us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":825150}
I20260618 13:59:34.023569 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ead386562eb4fd787c7aee776c2194f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.023630 28799 ts_tablet_manager.cc:1403] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260618 13:59:34.023852 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 52) took 1118 ms. Trace:
I20260618 13:59:34.023960 29745 rpcz_store.cc:276] 0618 13:59:32.905485 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.906710 (+ 1225us) service_pool.cc:224] Handling call
0618 13:59:34.023714 (+1117004us) ts_tablet_manager.cc:1151] Deleting tablet 897bf6d6bbee4ad89c296325d94a3e53
0618 13:59:34.023723 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:34.023783 (+ 60us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1116858}
I20260618 13:59:34.024300 29745 tablet_replica.cc:333] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:34.024415 28799 raft_consensus.cc:359] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.024515 29745 raft_consensus.cc:2243] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:34.024591 28799 raft_consensus.cc:385] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.024683 28799 raft_consensus.cc:740] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.024715 29745 raft_consensus.cc:2272] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:34.024897 28799 consensus_queue.cc:260] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.025233 29745 ts_tablet_manager.cc:1916] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:34.026575 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30f4bd3f86484caab48c1f0806299460" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.027006 28798 tablet_bootstrap.cc:654] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.025390 28799 ts_tablet_manager.cc:1434] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:34.028548 28799 tablet_bootstrap.cc:492] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:34.030589 28799 tablet_bootstrap.cc:654] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.030759 29400 raft_consensus.cc:493] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.030882 29400 raft_consensus.cc:515] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.031486 29400 leader_election.cc:290] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.031881 28617 leader_election.cc:304] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:34.032645 29303 raft_consensus.cc:2804] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:34.032770 29303 raft_consensus.cc:697] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:34.033202 29303 consensus_queue.cc:237] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.033843 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce7bf74b6d24d0bb43808b8a62cf1b7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.035703 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet fddea8fba4da40d7a5780e6750caf5e9 (table default.Table_4f1453c90c9f4b49b95607b12ad0bb65 [id=af94ae08478c4a37b443b5f46e4dd1c1]) successfully deleted
W20260618 13:59:34.035905 28346 leader_election.cc:343] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:34.036017 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 897bf6d6bbee4ad89c296325d94a3e53 because the tablet was not found. No further retry: Not found: Tablet not found: 897bf6d6bbee4ad89c296325d94a3e53
I20260618 13:59:34.036017 28346 leader_election.cc:304] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:34.036383 28346 leader_election.cc:343] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.036471 28346 leader_election.cc:304] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.037180 29914 raft_consensus.cc:2749] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:34.037358 29914 raft_consensus.cc:2749] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:34.035910 28799 tablet_bootstrap.cc:492] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:34.042614 28799 ts_tablet_manager.cc:1403] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.003s
W20260618 13:59:34.042622 28618 leader_election.cc:343] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.042888 29400 consensus_queue.cc:1048] T 72fd63e8245d4042b091593881ba11b6 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:34.043522 28799 raft_consensus.cc:359] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.043694 28799 raft_consensus.cc:385] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.043751 28799 raft_consensus.cc:740] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.043951 28799 consensus_queue.cc:260] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.044646 28799 ts_tablet_manager.cc:1434] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:34.044656 29714 consensus_queue.cc:1048] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:34.044902 28799 tablet_bootstrap.cc:492] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:34.046593 28799 tablet_bootstrap.cc:654] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.047614 28797 tablet_bootstrap.cc:492] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:34.047767 28797 ts_tablet_manager.cc:1403] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.000s
I20260618 13:59:34.048629 28797 raft_consensus.cc:359] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.048731 28595 heartbeater.cc:344] Connected to a master server at 127.27.100.60:42083
I20260618 13:59:34.049053 28797 raft_consensus.cc:385] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.049129 28797 raft_consensus.cc:740] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.049307 28797 consensus_queue.cc:260] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.050115 28797 ts_tablet_manager.cc:1434] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:34.050505 28797 tablet_bootstrap.cc:492] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:34.052193 28797 tablet_bootstrap.cc:654] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.054404 28797 tablet_bootstrap.cc:492] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:34.054544 28797 ts_tablet_manager.cc:1403] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20260618 13:59:34.055172 28797 raft_consensus.cc:359] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.055330 28797 raft_consensus.cc:385] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.055397 28797 raft_consensus.cc:740] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.055605 28797 consensus_queue.cc:260] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.056048 28797 ts_tablet_manager.cc:1434] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 13:59:34.056161 29566 ts_tablet_manager.cc:1950] T 316f470ea82f42fca48e4e96d49c9574 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:34.056329 28797 tablet_bootstrap.cc:492] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:34.057861 28797 tablet_bootstrap.cc:654] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.066174 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 50) took 2038 ms. Trace:
I20260618 13:59:34.066335 29566 rpcz_store.cc:276] 0618 13:59:32.027273 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.027350 (+ 77us) service_pool.cc:224] Handling call
0618 13:59:33.913038 (+1885688us) ts_tablet_manager.cc:1151] Deleting tablet 316f470ea82f42fca48e4e96d49c9574
0618 13:59:33.913049 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:34.015251 (+102202us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:34.018697 (+ 3446us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:34.066146 (+ 47449us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1885493}
I20260618 13:59:34.066812 29566 tablet_replica.cc:333] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:34.066982 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 316f470ea82f42fca48e4e96d49c9574 (table default.Table_28c746533f45405791fc12912e8fbcd4 [id=5415980263ed4fbea10b969e3b8facb4]) successfully deleted
I20260618 13:59:34.067049 29566 raft_consensus.cc:2243] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:34.067385 29566 raft_consensus.cc:2272] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:34.067948 29566 ts_tablet_manager.cc:1916] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:34.072170 29017 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_d31dc25d118d4cd2bb4dd8f034752827" } new_table_name: "default.Table_0158779f563348c78b3505f98d43b839" modify_external_catalogs: true
I20260618 13:59:34.076706 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce7bf74b6d24d0bb43808b8a62cf1b7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:34.077036 28411 raft_consensus.cc:2468] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:34.077524 28617 leader_election.cc:304] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:34.079577 29714 raft_consensus.cc:2804] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:34.079674 29714 raft_consensus.cc:493] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:34.079795 29714 raft_consensus.cc:3060] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.083231 29019 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 78) took 1006 ms. Trace:
I20260618 13:59:34.083362 29019 rpcz_store.cc:276] 0618 13:59:33.077169 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:33.265251 (+188082us) service_pool.cc:224] Handling call
0618 13:59:33.268383 (+ 3132us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb4686956 00007fcbab0e4146 00007fcbab0e3e75 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:33.268599 (+ 216us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:33.268817 (+ 218us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:33.269007 (+ 190us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:33.269073 (+ 66us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:33.305161 (+ 36088us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:34.083190 (+778029us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:33.300208 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:33.300322 (+ 114us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:33.300329 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:33.300331 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:33.300435 (+ 104us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:33.300448 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:33.300452 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:33.300513 (+ 61us) tablet.cc:701] Row locks acquired
0618 13:59:33.300515 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:33.300545 (+ 30us) write_op.cc:270] Start()
0618 13:59:33.300580 (+ 35us) write_op.cc:276] Timestamp: P: 1781791173300517 usec, L: 0
0618 13:59:33.300585 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:33.300755 (+ 170us) log.cc:844] Serialized 405 byte log entry
0618 13:59:33.304106 (+ 3351us) op_driver.cc:464] REPLICATION: finished
0618 13:59:33.304817 (+ 711us) write_op.cc:301] APPLY: starting
0618 13:59:33.304827 (+ 10us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:33.304855 (+ 28us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:33.304885 (+ 30us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:33.304918 (+ 33us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:33.304949 (+ 31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:33.304955 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:33.305004 (+ 49us) log.cc:844] Serialized 23 byte log entry
0618 13:59:33.305037 (+ 33us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:33.305050 (+ 13us) write_op.cc:454] Released schema lock
0618 13:59:33.305054 (+ 4us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:33.313166 (+ 8112us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:33.313174 (+ 8us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":6717184,"child_traces":[["op",{"apply.queue_time_us":681,"apply.run_cpu_time_us":285,"apply.run_wall_time_us":285,"num_ops":1,"prepare.queue_time_us":30748,"prepare.run_cpu_time_us":687,"prepare.run_wall_time_us":12945,"raft.queue_time_us":454,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"replication_time_us":3504}]]}
I20260618 13:59:34.084367 29679 raft_consensus.cc:493] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.084499 29679 raft_consensus.cc:515] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.085106 29679 leader_election.cc:290] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.085963 29019 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_72e5809d8cce4f3bbe8227efcb8e2b2c" } new_table_name: "default.Table_498fb462955c4472ae5aecef39e016cd" modify_external_catalogs: true
I20260618 13:59:34.088850 29007 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_0fbace1c1aa746a6a2a57c878631218b" } modify_external_catalogs: true
I20260618 13:59:34.099937 28457 heartbeater.cc:344] Connected to a master server at 127.27.100.60:42083
I20260618 13:59:34.101006 28798 tablet_bootstrap.cc:492] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
W20260618 13:59:34.101169 29017 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c881dd3cb66344969310956ad6e83cc0 consensus state has no leader
I20260618 13:59:34.101176 28798 ts_tablet_manager.cc:1403] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.119s user 0.003s sys 0.000s
I20260618 13:59:34.102044 29017 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_1a2d838221d44ca1952452c1475a0498"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:34.102134 28798 raft_consensus.cc:359] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.102344 28798 raft_consensus.cc:385] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.102502 28798 raft_consensus.cc:740] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
W20260618 13:59:34.102702 29017 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1a2d838221d44ca1952452c1475a0498 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:34.103026 28798 consensus_queue.cc:260] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.104310 28798 ts_tablet_manager.cc:1434] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260618 13:59:34.106689 29400 raft_consensus.cc:493] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.106848 29400 raft_consensus.cc:515] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.107470 29400 leader_election.cc:290] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.108111 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9bd8577f4f4a9593a3ffeab582b02e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:34.108337 28411 raft_consensus.cc:2468] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:34.108722 28617 leader_election.cc:304] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:34.109012 29400 raft_consensus.cc:2804] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:34.109105 29400 raft_consensus.cc:493] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:34.109174 29400 raft_consensus.cc:3060] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.115526 28729 heartbeater.cc:344] Connected to a master server at 127.27.100.60:42083
I20260618 13:59:34.119470 29746 ts_tablet_manager.cc:1929] T 9a0691de253b4b008e5d995fd6b876c2 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:34.119596 29746 log.cc:1199] T 9a0691de253b4b008e5d995fd6b876c2 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/9a0691de253b4b008e5d995fd6b876c2
I20260618 13:59:34.119971 29746 ts_tablet_manager.cc:1950] T 9a0691de253b4b008e5d995fd6b876c2 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:34.124285 29714 raft_consensus.cc:515] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.125200 29679 raft_consensus.cc:493] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.125412 29714 leader_election.cc:290] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.127398 28797 tablet_bootstrap.cc:492] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:34.127533 28797 ts_tablet_manager.cc:1403] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.071s user 0.006s sys 0.019s
I20260618 13:59:34.128427 28797 raft_consensus.cc:359] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.128604 28797 raft_consensus.cc:385] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.128669 28797 raft_consensus.cc:740] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.128955 28797 consensus_queue.cc:260] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.129482 28797 ts_tablet_manager.cc:1434] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:34.129751 28797 tablet_bootstrap.cc:492] T 3a0039a463ec4a9c82a21a3a69b565fc P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:34.131556 28797 tablet_bootstrap.cc:654] T 3a0039a463ec4a9c82a21a3a69b565fc P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.133107 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 9a0691de253b4b008e5d995fd6b876c2 (table default.Table_ab57b782a92747e78e2bfe28480aa0ec [id=28d43fd5660f459a9acfab2fc0dfbdb1]) successfully deleted
I20260618 13:59:34.134652 29679 raft_consensus.cc:515] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.134835 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce7bf74b6d24d0bb43808b8a62cf1b7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:34.134985 28403 raft_consensus.cc:3060] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.135428 29679 leader_election.cc:290] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.137240 28799 tablet_bootstrap.cc:492] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:34.137367 28799 ts_tablet_manager.cc:1403] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.093s user 0.007s sys 0.019s
I20260618 13:59:34.138175 28799 raft_consensus.cc:359] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.138321 28799 raft_consensus.cc:385] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.138375 28799 raft_consensus.cc:740] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.138607 28799 consensus_queue.cc:260] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.139242 28799 ts_tablet_manager.cc:1434] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:34.139529 28799 tablet_bootstrap.cc:492] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:34.141222 28799 tablet_bootstrap.cc:654] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.150008 29017 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 83) took 1008 ms. Trace:
I20260618 13:59:34.150275 29017 rpcz_store.cc:276] 0618 13:59:33.141498 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.101831 (+960333us) service_pool.cc:224] Handling call
0618 13:59:34.113723 (+ 11892us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:34.114218 (+ 495us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:34.149754 (+ 35536us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:34.149858 (+ 104us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:34.149946 (+ 88us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:34.141744 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:34.141900 (+ 156us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:34.141905 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:34.141907 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:34.142040 (+ 133us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:34.142056 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:34.142064 (+ 8us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:34.142174 (+ 110us) tablet.cc:701] Row locks acquired
0618 13:59:34.142178 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:34.142215 (+ 37us) write_op.cc:270] Start()
0618 13:59:34.142252 (+ 37us) write_op.cc:276] Timestamp: P: 1781791174142150 usec, L: 0
0618 13:59:34.142258 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:34.142510 (+ 252us) log.cc:844] Serialized 433 byte log entry
0618 13:59:34.146788 (+ 4278us) op_driver.cc:464] REPLICATION: finished
0618 13:59:34.146890 (+ 102us) write_op.cc:301] APPLY: starting
0618 13:59:34.146903 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:34.146960 (+ 57us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:34.146999 (+ 39us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:34.147161 (+ 162us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:34.147206 (+ 45us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:34.147217 (+ 11us) write_op.cc:312] APPLY: finished
0618 13:59:34.147299 (+ 82us) log.cc:844] Serialized 29 byte log entry
0618 13:59:34.147368 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.147389 (+ 21us) write_op.cc:454] Released schema lock
0618 13:59:34.147411 (+ 22us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:34.147493 (+ 82us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.147499 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":56,"apply.run_cpu_time_us":587,"apply.run_wall_time_us":587,"mutex_wait_us":15383,"num_ops":2,"prepare.queue_time_us":15431,"prepare.run_cpu_time_us":895,"prepare.run_wall_time_us":902,"raft.queue_time_us":80,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":263,"replication_time_us":4500}]]}
I20260618 13:59:34.152169 29017 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_2dc7e2d92c504d7f91ac02de7c2bdee4" } new_table_name: "default.Table_b92d4faf69a64f5787f610c8f1ecca22" modify_external_catalogs: true
I20260618 13:59:34.156307 28799 tablet_bootstrap.cc:492] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:34.156441 28799 ts_tablet_manager.cc:1403] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.002s
I20260618 13:59:34.157264 28799 raft_consensus.cc:359] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.157426 28799 raft_consensus.cc:385] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.157480 28799 raft_consensus.cc:740] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.157708 28799 consensus_queue.cc:260] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.158351 28799 ts_tablet_manager.cc:1434] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 13:59:34.158655 28799 tablet_bootstrap.cc:492] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:34.160290 28799 tablet_bootstrap.cc:654] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.164315 29025 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_c70bc1c98b3b49d7aa3080904667732d [id=901874cb80e74ba589679bb7b95c8fe4] alter complete (version 1)
I20260618 13:59:34.133092 29746 tablet_replica.cc:333] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:34.174767 29746 raft_consensus.cc:2243] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:34.174940 29746 raft_consensus.cc:2272] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:34.175495 29746 ts_tablet_manager.cc:1916] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:34.168854 29759 consensus_queue.cc:1048] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260618 13:59:34.172097 28596 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:34.183634 28799 tablet_bootstrap.cc:492] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:34.183754 28799 ts_tablet_manager.cc:1403] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.000s
I20260618 13:59:34.184538 28799 raft_consensus.cc:359] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.184711 28799 raft_consensus.cc:385] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.184770 28799 raft_consensus.cc:740] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.184953 28799 consensus_queue.cc:260] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.185437 28799 ts_tablet_manager.cc:1434] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:34.185719 28799 tablet_bootstrap.cc:492] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:34.188158 28799 tablet_bootstrap.cc:654] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.196755 29749 raft_consensus.cc:493] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.196938 29749 raft_consensus.cc:515] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:34.197652 29749 leader_election.cc:290] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:34.198312 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea4bc13eabc4dd98b4f96723498014c" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:34.202729 28458 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:34.211346 28797 tablet_bootstrap.cc:492] T 3a0039a463ec4a9c82a21a3a69b565fc P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:34.211494 28797 ts_tablet_manager.cc:1403] T 3a0039a463ec4a9c82a21a3a69b565fc P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.082s user 0.005s sys 0.025s
I20260618 13:59:34.212311 28797 raft_consensus.cc:359] T 3a0039a463ec4a9c82a21a3a69b565fc P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.212499 28797 raft_consensus.cc:385] T 3a0039a463ec4a9c82a21a3a69b565fc P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.212553 28797 raft_consensus.cc:740] T 3a0039a463ec4a9c82a21a3a69b565fc P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.212759 28797 consensus_queue.cc:260] T 3a0039a463ec4a9c82a21a3a69b565fc P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.213243 28797 ts_tablet_manager.cc:1434] T 3a0039a463ec4a9c82a21a3a69b565fc P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:34.213529 28797 tablet_bootstrap.cc:492] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:34.215174 28797 tablet_bootstrap.cc:654] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.128556 28798 tablet_bootstrap.cc:492] T 6eea4d59935d40aa97dc10ec7469aa56 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:34.218526 29679 raft_consensus.cc:493] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.218674 29679 raft_consensus.cc:515] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.219167 29400 raft_consensus.cc:515] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.219415 29679 leader_election.cc:290] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.219746 29400 leader_election.cc:290] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.219997 29679 raft_consensus.cc:493] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.220105 29679 raft_consensus.cc:515] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.220357 29745 ts_tablet_manager.cc:1929] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:34.220446 29745 log.cc:1199] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/f89c9954d48e407c96c294c26644e25c
I20260618 13:59:34.220587 29679 leader_election.cc:290] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.220759 29745 ts_tablet_manager.cc:1950] T f89c9954d48e407c96c294c26644e25c P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:34.221545 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9bd8577f4f4a9593a3ffeab582b02e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:34.221730 29566 ts_tablet_manager.cc:1929] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:34.221730 28409 raft_consensus.cc:3060] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.221815 29566 log.cc:1199] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/2188ebf1727240cd886ba7662f8fcf43
I20260618 13:59:34.222137 29566 ts_tablet_manager.cc:1950] T 2188ebf1727240cd886ba7662f8fcf43 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:34.224367 28403 raft_consensus.cc:2468] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:34.227711 28617 leader_election.cc:304] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:34.228366 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 53) took 1322 ms. Trace:
I20260618 13:59:34.228467 29745 rpcz_store.cc:276] 0618 13:59:32.905575 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.906884 (+ 1309us) service_pool.cc:224] Handling call
0618 13:59:34.024172 (+1117288us) ts_tablet_manager.cc:1151] Deleting tablet f89c9954d48e407c96c294c26644e25c
0618 13:59:34.024182 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:34.120425 (+ 96243us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:34.220289 (+ 99864us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:34.228345 (+ 8056us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1117111}
I20260618 13:59:34.228796 29644 raft_consensus.cc:2804] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:34.228870 29745 tablet_replica.cc:333] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:34.229096 29745 raft_consensus.cc:2243] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:34.229238 29745 raft_consensus.cc:2272] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:34.229542 29644 raft_consensus.cc:697] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:34.229748 29745 ts_tablet_manager.cc:1916] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:34.235225 29019 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 60b6216474b74a14af6abf977e312b77 consensus state has no leader
I20260618 13:59:34.235414 29019 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 82) took 1106 ms. Trace:
I20260618 13:59:34.235527 29019 rpcz_store.cc:276] 0618 13:59:33.129145 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.085757 (+956612us) service_pool.cc:224] Handling call
0618 13:59:34.086236 (+ 479us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:34.086518 (+ 282us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:34.086740 (+ 222us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:34.086829 (+ 89us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:34.102828 (+ 15999us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:34.235372 (+132544us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:34.087449 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:34.087564 (+ 115us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:34.087571 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:34.087573 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:34.087703 (+ 130us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:34.087719 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:34.087726 (+ 7us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:34.087801 (+ 75us) tablet.cc:701] Row locks acquired
0618 13:59:34.087805 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:34.087841 (+ 36us) write_op.cc:270] Start()
0618 13:59:34.087878 (+ 37us) write_op.cc:276] Timestamp: P: 1781791174087782 usec, L: 0
0618 13:59:34.087884 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:34.088081 (+ 197us) log.cc:844] Serialized 405 byte log entry
0618 13:59:34.096257 (+ 8176us) op_driver.cc:464] REPLICATION: finished
0618 13:59:34.096364 (+ 107us) write_op.cc:301] APPLY: starting
0618 13:59:34.096385 (+ 21us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:34.096453 (+ 68us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:34.096503 (+ 50us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:34.096561 (+ 58us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:34.096603 (+ 42us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:34.096614 (+ 11us) write_op.cc:312] APPLY: finished
0618 13:59:34.096700 (+ 86us) log.cc:844] Serialized 23 byte log entry
0618 13:59:34.096763 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.096781 (+ 18us) write_op.cc:454] Released schema lock
0618 13:59:34.096787 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:34.096853 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.096857 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":58,"apply.run_cpu_time_us":481,"apply.run_wall_time_us":481,"num_ops":1,"prepare.queue_time_us":52,"prepare.run_cpu_time_us":723,"prepare.run_wall_time_us":723,"replication_time_us":8341}]]}
I20260618 13:59:34.237025 29019 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_a8705c5cd49a40049ff1020a01aa7f5b" } modify_external_catalogs: true
I20260618 13:59:34.237270 29019 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_a8705c5cd49a40049ff1020a01aa7f5b" } modify_external_catalogs: true
I20260618 13:59:34.237846 29644 consensus_queue.cc:237] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.243069 28797 tablet_bootstrap.cc:492] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:34.243278 28797 ts_tablet_manager.cc:1403] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.005s
I20260618 13:59:34.244220 28797 raft_consensus.cc:359] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.244422 28797 raft_consensus.cc:385] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.244475 28797 raft_consensus.cc:740] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.245829 28797 consensus_queue.cc:260] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.246862 28797 ts_tablet_manager.cc:1434] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:34.247150 28797 tablet_bootstrap.cc:492] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:34.248888 28797 tablet_bootstrap.cc:654] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.256547 29007 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 962bf304a285412fb2a529db5b994e9e
I20260618 13:59:34.257294 29963 log.cc:826] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Log is configured to *not* fsync() on all Append() calls
I20260618 13:59:34.257513 29007 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 80) took 1153 ms. Trace:
I20260618 13:59:34.257774 29007 rpcz_store.cc:276] 0618 13:59:33.103722 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:33.974178 (+870456us) service_pool.cc:224] Handling call
0618 13:59:34.089012 (+114834us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:34.089202 (+ 190us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:34.089270 (+ 68us) catalog_manager.cc:2849] Locking tablets
0618 13:59:34.247357 (+158087us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:34.256348 (+ 8991us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:34.256435 (+ 87us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:34.256471 (+ 36us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:34.257472 (+ 1001us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:34.249365 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:34.249541 (+ 176us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:34.249550 (+ 9us) write_op.cc:435] Acquired schema lock
0618 13:59:34.249554 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:34.249700 (+ 146us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:34.249717 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:34.249725 (+ 8us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:34.249862 (+ 137us) tablet.cc:701] Row locks acquired
0618 13:59:34.249866 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:34.249900 (+ 34us) write_op.cc:270] Start()
0618 13:59:34.249935 (+ 35us) write_op.cc:276] Timestamp: P: 1781791174249884 usec, L: 0
0618 13:59:34.249941 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:34.250152 (+ 211us) log.cc:844] Serialized 728 byte log entry
0618 13:59:34.254840 (+ 4688us) op_driver.cc:464] REPLICATION: finished
0618 13:59:34.254959 (+ 119us) write_op.cc:301] APPLY: starting
0618 13:59:34.254978 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:34.255036 (+ 58us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:34.255079 (+ 43us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:34.255170 (+ 91us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:34.255214 (+ 44us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:34.255225 (+ 11us) write_op.cc:312] APPLY: finished
0618 13:59:34.255312 (+ 87us) log.cc:844] Serialized 29 byte log entry
0618 13:59:34.255378 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.255416 (+ 38us) write_op.cc:454] Released schema lock
0618 13:59:34.255426 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:34.255506 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.255513 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":64,"apply.run_cpu_time_us":535,"apply.run_wall_time_us":535,"num_ops":2,"prepare.queue_time_us":1175,"prepare.run_cpu_time_us":922,"prepare.run_wall_time_us":1679,"raft.queue_time_us":39,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":258,"replication_time_us":4866}]]}
I20260618 13:59:34.259994 29007 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_84a1abff52a348629514e93e578bfac3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:34.262106 28799 tablet_bootstrap.cc:492] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:34.262269 28799 ts_tablet_manager.cc:1403] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.077s user 0.018s sys 0.003s
I20260618 13:59:34.263132 28799 raft_consensus.cc:359] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.263347 28799 raft_consensus.cc:385] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.263397 28799 raft_consensus.cc:740] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.263597 28799 consensus_queue.cc:260] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.264075 28799 ts_tablet_manager.cc:1434] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:34.264336 28799 tablet_bootstrap.cc:492] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:34.268411 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 51) took 2240 ms. Trace:
I20260618 13:59:34.268509 29566 rpcz_store.cc:276] 0618 13:59:32.028291 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.028370 (+ 79us) service_pool.cc:224] Handling call
0618 13:59:34.066751 (+2038381us) ts_tablet_manager.cc:1151] Deleting tablet 2188ebf1727240cd886ba7662f8fcf43
0618 13:59:34.066761 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:34.138248 (+ 71487us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:34.221670 (+ 83422us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:34.268388 (+ 46718us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2038209}
I20260618 13:59:34.268878 29566 tablet_replica.cc:333] T 3a0039a463ec4a9c82a21a3a69b565fc P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:34.269105 28799 tablet_bootstrap.cc:654] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.270640 29566 raft_consensus.cc:2243] T 3a0039a463ec4a9c82a21a3a69b565fc P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:34.270812 29566 raft_consensus.cc:2272] T 3a0039a463ec4a9c82a21a3a69b565fc P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:34.270833 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 962bf304a285412fb2a529db5b994e9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:34 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.271373 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 962bf304a285412fb2a529db5b994e9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:34 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.271533 29566 ts_tablet_manager.cc:1916] T 3a0039a463ec4a9c82a21a3a69b565fc P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:34.271905 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 2188ebf1727240cd886ba7662f8fcf43 (table default.Table_fac3fdd9b466497683bc00300a85fed1 [id=a4ac38531af34b4681abba96b42ffb7f]) successfully deleted
W20260618 13:59:34.273288 29007 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_84a1abff52a348629514e93e578bfac3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:34.278620 28988 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 9024952ab8094fcda1329562bbde8ef5 consensus state has no leader
I20260618 13:59:34.287359 28798 tablet_bootstrap.cc:654] T 6eea4d59935d40aa97dc10ec7469aa56 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.288347 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9bd8577f4f4a9593a3ffeab582b02e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.289363 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce7bf74b6d24d0bb43808b8a62cf1b7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:34.291080 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9bd8577f4f4a9593a3ffeab582b02e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:34.292057 28346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 255655us
I20260618 13:59:34.293112 28797 tablet_bootstrap.cc:492] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:34.293259 28797 ts_tablet_manager.cc:1403] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.046s user 0.001s sys 0.013s
I20260618 13:59:34.294014 28797 raft_consensus.cc:359] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.294173 28797 raft_consensus.cc:385] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.294224 28797 raft_consensus.cc:740] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.294473 28797 consensus_queue.cc:260] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.295008 28797 ts_tablet_manager.cc:1434] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:34.295255 28797 tablet_bootstrap.cc:492] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:34.295392 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea4bc13eabc4dd98b4f96723498014c" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.297992 28797 tablet_bootstrap.cc:654] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.298403 28595 heartbeater.cc:344] Connected to a master server at 127.27.100.60:42083
I20260618 13:59:34.304517 28797 tablet_bootstrap.cc:492] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:34.304646 28797 ts_tablet_manager.cc:1403] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.001s
I20260618 13:59:34.305938 28548 raft_consensus.cc:3060] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.306818 28797 raft_consensus.cc:359] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.307003 28797 raft_consensus.cc:385] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.307049 28797 raft_consensus.cc:740] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.307224 28797 consensus_queue.cc:260] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.307667 28797 ts_tablet_manager.cc:1434] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:34.307930 28797 tablet_bootstrap.cc:492] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:34.309700 28797 tablet_bootstrap.cc:654] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.311733 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2ee746c5024cfc9b31c470f9616e7d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.312193 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c92a594cd64982a8972e4a019d1cd8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.312237 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2ee746c5024cfc9b31c470f9616e7d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.312906 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c92a594cd64982a8972e4a019d1cd8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.313068 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d8dd2c388e43a0be380aa953ec2c82" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.314064 29984 raft_consensus.cc:493] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.314217 29984 raft_consensus.cc:515] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.314304 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab1ddf79b094ba2aeee9225c0b332c3" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.314920 29984 leader_election.cc:290] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.315143 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 962bf304a285412fb2a529db5b994e9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:34 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:34.315490 28409 raft_consensus.cc:2468] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:34.315523 28797 tablet_bootstrap.cc:492] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:34.315702 28797 ts_tablet_manager.cc:1403] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260618 13:59:34.316361 28797 raft_consensus.cc:359] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.316505 28797 raft_consensus.cc:385] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.316550 28797 raft_consensus.cc:740] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.316753 28797 consensus_queue.cc:260] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.319375 28797 ts_tablet_manager.cc:1434] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:34.319557 29746 ts_tablet_manager.cc:1929] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:34.319666 29746 log.cc:1199] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/98b42d23832545ae81a7cdff19e4689f
I20260618 13:59:34.320009 29746 ts_tablet_manager.cc:1950] T 98b42d23832545ae81a7cdff19e4689f P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:34.321372 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab1ddf79b094ba2aeee9225c0b332c3" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.322047 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d8dd2c388e43a0be380aa953ec2c82" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.325537 29914 raft_consensus.cc:493] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.325649 28617 leader_election.cc:304] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:34.325690 29914 raft_consensus.cc:515] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.325943 29679 raft_consensus.cc:493] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.326058 29679 raft_consensus.cc:515] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.326267 29914 leader_election.cc:290] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.326581 29679 leader_election.cc:290] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.326730 28799 tablet_bootstrap.cc:492] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:34.326850 28799 ts_tablet_manager.cc:1403] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.063s user 0.004s sys 0.017s
I20260618 13:59:34.326853 29719 raft_consensus.cc:2804] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:34.326990 29719 raft_consensus.cc:697] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:34.327193 29719 consensus_queue.cc:237] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.327557 28799 raft_consensus.cc:359] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.327694 28799 raft_consensus.cc:385] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.327740 28799 raft_consensus.cc:740] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.327907 28799 consensus_queue.cc:260] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.328248 28799 ts_tablet_manager.cc:1434] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 13:59:34.328490 28799 tablet_bootstrap.cc:492] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:34.329105 28484 leader_election.cc:304] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.329595 29755 consensus_queue.cc:1048] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:34.330232 28799 tablet_bootstrap.cc:654] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:34.328192 28618 leader_election.cc:343] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:34.331401 28618 leader_election.cc:343] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.331578 28544 raft_consensus.cc:3060] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:34.331826 28618 consensus_peers.cc:597] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:34.331979 28618 consensus_peers.cc:597] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:34.332086 28618 leader_election.cc:343] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.335765 29644 raft_consensus.cc:493] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.335881 29644 raft_consensus.cc:515] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.339078 29644 leader_election.cc:290] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.339808 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a10e83d26b704201b95e7c0c44a1a4f5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:34.340041 28409 raft_consensus.cc:2468] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:34.340564 28617 leader_election.cc:304] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:34.340951 29644 raft_consensus.cc:2804] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:34.341033 29644 raft_consensus.cc:493] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:34.341085 29644 raft_consensus.cc:3060] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.351022 29755 raft_consensus.cc:493] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.351159 29755 raft_consensus.cc:515] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.351807 29755 leader_election.cc:290] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:34.353523 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d014b46a0f448b929cbc9287ead0a7" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
W20260618 13:59:34.354357 28346 leader_election.cc:343] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:34.354543 28346 leader_election.cc:343] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:34.354763 28346 leader_election.cc:343] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:34.354907 28346 leader_election.cc:343] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.356081 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5e04eb43ed4cf493c0f6f08e22baeb" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.356714 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e25027f423144b088dad955c7282921" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.356984 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ba9186b3ff47d5a2b158fef619686e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.357319 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5e04eb43ed4cf493c0f6f08e22baeb" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.357623 28539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e25027f423144b088dad955c7282921" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.357890 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ba9186b3ff47d5a2b158fef619686e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.362767 29749 raft_consensus.cc:2749] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:34.362730 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a10e83d26b704201b95e7c0c44a1a4f5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.364152 28797 tablet_bootstrap.cc:492] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:34.364519 28618 leader_election.cc:343] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.365840 28797 tablet_bootstrap.cc:654] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.365877 28799 tablet_bootstrap.cc:492] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:34.365996 28799 ts_tablet_manager.cc:1403] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.038s user 0.001s sys 0.007s
I20260618 13:59:34.366173 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d014b46a0f448b929cbc9287ead0a7" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.366984 28484 leader_election.cc:304] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.371225 29749 raft_consensus.cc:2749] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:34.371310 28798 tablet_bootstrap.cc:492] T 6eea4d59935d40aa97dc10ec7469aa56 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:34.371214 28799 raft_consensus.cc:359] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.371376 28799 raft_consensus.cc:385] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.371417 28798 ts_tablet_manager.cc:1403] T 6eea4d59935d40aa97dc10ec7469aa56 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.243s user 0.003s sys 0.000s
I20260618 13:59:34.371428 28799 raft_consensus.cc:740] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.371650 28799 consensus_queue.cc:260] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.372149 28799 ts_tablet_manager.cc:1434] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260618 13:59:34.372139 28798 raft_consensus.cc:359] T 6eea4d59935d40aa97dc10ec7469aa56 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:34.372303 28798 raft_consensus.cc:385] T 6eea4d59935d40aa97dc10ec7469aa56 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.372359 28798 raft_consensus.cc:740] T 6eea4d59935d40aa97dc10ec7469aa56 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.372414 28799 tablet_bootstrap.cc:492] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:34.372526 28798 consensus_queue.cc:260] T 6eea4d59935d40aa97dc10ec7469aa56 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:34.373026 28798 ts_tablet_manager.cc:1434] T 6eea4d59935d40aa97dc10ec7469aa56 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
W20260618 13:59:34.375064 28346 leader_election.cc:343] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.375186 28346 leader_election.cc:304] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.376080 29984 raft_consensus.cc:2749] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:34.376076 28346 leader_election.cc:343] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.381173 28346 leader_election.cc:304] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:34.382212 28346 leader_election.cc:343] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.382326 28346 leader_election.cc:304] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.385069 29679 raft_consensus.cc:2749] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:34.385249 29679 raft_consensus.cc:2749] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:34.386284 28799 tablet_bootstrap.cc:654] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:34.390614 29017 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c25fa765ec794c34b0aa43761961a8c9 consensus state has no leader
I20260618 13:59:34.390738 29007 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 86) took 1101 ms. Trace:
W20260618 13:59:34.386240 28346 leader_election.cc:343] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.399083 28410 raft_consensus.cc:1275] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Refusing update from remote peer d6a169b16ced46ca9cfbc631709bb257: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 13:59:34.402253 29303 raft_consensus.cc:493] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.402407 29303 raft_consensus.cc:515] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.403107 29303 leader_election.cc:290] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.403934 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450c68da26404767b01c5e8da3f6286d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:34.404548 28618 leader_election.cc:343] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.417644 28798 tablet_bootstrap.cc:492] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:34.422220 29566 ts_tablet_manager.cc:1929] T 3a0039a463ec4a9c82a21a3a69b565fc P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:34.422228 29745 ts_tablet_manager.cc:1929] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:34.422343 29566 log.cc:1199] T 3a0039a463ec4a9c82a21a3a69b565fc P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/3a0039a463ec4a9c82a21a3a69b565fc
I20260618 13:59:34.422345 29745 log.cc:1199] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/dd2385e6752d49429422fc854539e9c6
I20260618 13:59:34.422631 29566 ts_tablet_manager.cc:1950] T 3a0039a463ec4a9c82a21a3a69b565fc P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:34.423110 29644 raft_consensus.cc:515] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.423666 29644 leader_election.cc:290] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.427399 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a10e83d26b704201b95e7c0c44a1a4f5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:34.428000 28618 leader_election.cc:343] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.428438 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 52) took 2398 ms. Trace:
I20260618 13:59:34.428534 29566 rpcz_store.cc:276] 0618 13:59:32.029523 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.029582 (+ 59us) service_pool.cc:224] Handling call
0618 13:59:34.268784 (+2239202us) ts_tablet_manager.cc:1151] Deleting tablet 3a0039a463ec4a9c82a21a3a69b565fc
0618 13:59:34.268804 (+ 20us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:34.330982 (+ 62178us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:34.422154 (+ 91172us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:34.428422 (+ 6268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1552,"tablet-delete.queue_time_us":2238875}
I20260618 13:59:34.428870 29566 tablet_replica.cc:333] T 8fd9fa9675234db2abff694a517c0ea8 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:34.418773 29007 rpcz_store.cc:276] 0618 13:59:33.184606 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.259733 (+1075127us) service_pool.cc:224] Handling call
0618 13:59:34.273463 (+ 13730us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:34.274007 (+ 544us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:34.285977 (+ 11970us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:34.286086 (+ 109us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:34.286181 (+ 95us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:34.274583 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:34.274729 (+ 146us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:34.274734 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:34.274737 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:34.274826 (+ 89us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:34.274840 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:34.274844 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:34.274924 (+ 80us) tablet.cc:701] Row locks acquired
0618 13:59:34.274927 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:34.274965 (+ 38us) write_op.cc:270] Start()
0618 13:59:34.275004 (+ 39us) write_op.cc:276] Timestamp: P: 1781791174274943 usec, L: 0
0618 13:59:34.275009 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:34.275191 (+ 182us) log.cc:844] Serialized 433 byte log entry
0618 13:59:34.280590 (+ 5399us) op_driver.cc:464] REPLICATION: finished
0618 13:59:34.280717 (+ 127us) write_op.cc:301] APPLY: starting
0618 13:59:34.280735 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:34.280794 (+ 59us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:34.280831 (+ 37us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:34.280960 (+ 129us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:34.281004 (+ 44us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:34.281014 (+ 10us) write_op.cc:312] APPLY: finished
0618 13:59:34.281105 (+ 91us) log.cc:844] Serialized 29 byte log entry
0618 13:59:34.281171 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.281192 (+ 21us) write_op.cc:454] Released schema lock
0618 13:59:34.281201 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:34.281317 (+ 116us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.281321 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":64,"apply.run_cpu_time_us":558,"apply.run_wall_time_us":557,"num_ops":2,"prepare.queue_time_us":51,"prepare.run_cpu_time_us":717,"prepare.run_wall_time_us":1379,"raft.queue_time_us":51,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":277,"replication_time_us":5552}]]}
I20260618 13:59:34.429042 29566 raft_consensus.cc:2243] T 8fd9fa9675234db2abff694a517c0ea8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:34.429189 29566 raft_consensus.cc:2272] T 8fd9fa9675234db2abff694a517c0ea8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:34.430068 29566 ts_tablet_manager.cc:1916] T 8fd9fa9675234db2abff694a517c0ea8 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:34.397712 28346 leader_election.cc:304] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.432412 29679 raft_consensus.cc:2749] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:34.432835 28798 tablet_bootstrap.cc:654] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:34.433956 28346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 47418.8us
I20260618 13:59:34.435909 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450c68da26404767b01c5e8da3f6286d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:34.436088 28409 raft_consensus.cc:2468] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:34.436441 29644 raft_consensus.cc:493] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.436558 29644 raft_consensus.cc:515] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:34.436417 28346 leader_election.cc:343] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.437441 29644 leader_election.cc:290] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:34.438050 28346 leader_election.cc:343] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.438987 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2df9e9e811ec4c2ea68467b1294bb62e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:34.439581 28618 leader_election.cc:343] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.439857 29019 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 35d8dd2c388e43a0be380aa953ec2c82
I20260618 13:59:34.440335 29017 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 84) took 1252 ms. Trace:
I20260618 13:59:34.440508 29017 rpcz_store.cc:276] 0618 13:59:33.182804 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.151838 (+969034us) service_pool.cc:224] Handling call
0618 13:59:34.152439 (+ 601us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:34.152690 (+ 251us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:34.152907 (+ 217us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:34.152986 (+ 79us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:34.310617 (+157631us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.310626 (+ 9us) write_op.cc:454] Released schema lock
0618 13:59:34.310730 (+ 104us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:34.435530 (+124800us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:34.153560 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:34.153684 (+ 124us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:34.153712 (+ 28us) write_op.cc:435] Acquired schema lock
0618 13:59:34.153716 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:34.153830 (+ 114us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:34.153845 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:34.153852 (+ 7us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:34.153976 (+ 124us) tablet.cc:701] Row locks acquired
0618 13:59:34.153980 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:34.154014 (+ 34us) write_op.cc:270] Start()
0618 13:59:34.154048 (+ 34us) write_op.cc:276] Timestamp: P: 1781791174153946 usec, L: 0
0618 13:59:34.154054 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:34.154255 (+ 201us) log.cc:844] Serialized 405 byte log entry
0618 13:59:34.158286 (+ 4031us) op_driver.cc:464] REPLICATION: finished
0618 13:59:34.158394 (+ 108us) write_op.cc:301] APPLY: starting
0618 13:59:34.158442 (+ 48us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:34.158478 (+ 36us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:34.158511 (+ 33us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:34.158555 (+ 44us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:34.158585 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:34.158593 (+ 8us) write_op.cc:312] APPLY: finished
0618 13:59:34.158655 (+ 62us) log.cc:844] Serialized 23 byte log entry
0618 13:59:34.158712 (+ 57us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.158727 (+ 15us) write_op.cc:454] Released schema lock
0618 13:59:34.158732 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":44711,"child_traces":[["op",{"apply.queue_time_us":63,"apply.run_cpu_time_us":386,"apply.run_wall_time_us":384,"num_ops":1,"prepare.queue_time_us":52,"prepare.run_cpu_time_us":760,"prepare.run_wall_time_us":791,"raft.queue_time_us":599,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":252,"replication_time_us":4205}]]}
I20260618 13:59:34.442296 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet f89c9954d48e407c96c294c26644e25c (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]) successfully deleted
I20260618 13:59:34.442874 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 98b42d23832545ae81a7cdff19e4689f (table default.Table_c99584ecc88948a097b39c22d68b3938 [id=7c39b3c432e944bca05f7b2070a79ff9]) successfully deleted
I20260618 13:59:34.443255 29721 raft_consensus.cc:493] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.443389 29721 raft_consensus.cc:515] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.444015 29721 leader_election.cc:290] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.444391 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "015494374645469785f1f978eaf0e9a5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.444734 29017 master_service.cc:960] ReplaceTablet: received request to replace tablet 9c17e76ae9b44207998a43e3576908c4 from {username='slave'} at 127.0.0.1:33144
W20260618 13:59:34.444940 28618 leader_election.cc:343] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.454803 29644 raft_consensus.cc:493] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.454984 29644 raft_consensus.cc:515] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:34.455667 29644 leader_election.cc:290] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.456667 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be00bfa9bc1a4b83b18bfad05ba1495e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:34.457113 28618 leader_election.cc:343] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.459232 29714 raft_consensus.cc:493] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.459339 29714 raft_consensus.cc:515] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.459759 29714 leader_election.cc:290] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.461103 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4cfca9e76e641a7bc0b58b9ee240f36" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.461548 29759 raft_consensus.cc:493] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:34.461663 29759 raft_consensus.cc:515] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.462086 28730 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:34.462304 29759 leader_election.cc:290] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:34.463131 28618 leader_election.cc:343] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.464154 28798 tablet_bootstrap.cc:492] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:34.464352 28798 ts_tablet_manager.cc:1403] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.054s user 0.003s sys 0.003s
W20260618 13:59:34.465734 28484 leader_election.cc:343] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 233040a16b0d4b61b977ca57c1afec2d
W20260618 13:59:34.466301 28618 consensus_peers.cc:597] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 98b42d23832545ae81a7cdff19e4689f. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:34.466584 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 962bf304a285412fb2a529db5b994e9e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 962bf304a285412fb2a529db5b994e9e already in progress: creating tablet
I20260618 13:59:34.468029 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 3a0039a463ec4a9c82a21a3a69b565fc (table default.Table_8c1fdd54393c4ad383e72a07837ead2a [id=6a7e8e3becb64fd5bc583ee08eaa4437]) successfully deleted
I20260618 13:59:34.468711 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet e5f7eaa7d218440ca2c35340c5c87814 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.469012 29749 raft_consensus.cc:493] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.469050 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 97a9f949999446f09359629fe483bf67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:29 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.469188 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet da4d4f03911f4ac099ecc2fa1d8af494 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.469117 29749 raft_consensus.cc:515] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:34.469316 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet aa8cf927bed9437086efd8e0f21ba26a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.469671 29749 leader_election.cc:290] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:34.469884 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 35d8dd2c388e43a0be380aa953ec2c82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:34 UTC) from {username='slave'} at 127.0.0.1:57282
W20260618 13:59:34.471519 28484 leader_election.cc:343] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: e025b64380314546af8a1ab90c817825
I20260618 13:59:34.472224 28663 tablet_service.cc:1558] Processing DeleteTablet for tablet 35d8dd2c388e43a0be380aa953ec2c82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:34 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.473125 28798 raft_consensus.cc:359] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.473523 28798 raft_consensus.cc:385] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.473606 28798 raft_consensus.cc:740] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.474011 28798 consensus_queue.cc:260] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.475411 29746 tablet_replica.cc:333] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:34.475626 29746 raft_consensus.cc:2243] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:34.475960 29679 raft_consensus.cc:493] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.476110 29679 raft_consensus.cc:515] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.476346 29746 raft_consensus.cc:2272] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:34.477120 29746 ts_tablet_manager.cc:1916] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:34.479718 28798 ts_tablet_manager.cc:1434] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.015s user 0.000s sys 0.003s
I20260618 13:59:34.482035 28798 tablet_bootstrap.cc:492] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
W20260618 13:59:34.501268 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet e5f7eaa7d218440ca2c35340c5c87814 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5f7eaa7d218440ca2c35340c5c87814 already in progress: creating tablet
W20260618 13:59:34.501789 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 97a9f949999446f09359629fe483bf67 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 97a9f949999446f09359629fe483bf67 already in progress: creating tablet
I20260618 13:59:34.503002 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fae928833e44d2ba860d6184d4c0142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.503584 28798 tablet_bootstrap.cc:654] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.507292 30019 tablet.cc:1724] T 39b06ae7444a48dbbec9e096fd7e4052 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:34.511075 29019 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 85) took 1282 ms. Trace:
I20260618 13:59:34.511354 29019 rpcz_store.cc:276] 0618 13:59:33.182978 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.236824 (+1053846us) service_pool.cc:224] Handling call
0618 13:59:34.237397 (+ 573us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:34.237560 (+ 163us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:34.237615 (+ 55us) catalog_manager.cc:2849] Locking tablets
0618 13:59:34.275122 (+ 37507us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:34.392854 (+117732us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:34.392956 (+ 102us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:34.393003 (+ 47us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:34.465714 (+ 72711us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:34.276013 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:34.276136 (+ 123us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:34.276143 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:34.276147 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:34.276263 (+ 116us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:34.276275 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:34.276282 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:34.276389 (+ 107us) tablet.cc:701] Row locks acquired
0618 13:59:34.276391 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:34.276421 (+ 30us) write_op.cc:270] Start()
0618 13:59:34.276451 (+ 30us) write_op.cc:276] Timestamp: P: 1781791174276400 usec, L: 0
0618 13:59:34.276457 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:34.276616 (+ 159us) log.cc:844] Serialized 728 byte log entry
0618 13:59:34.385727 (+109111us) op_driver.cc:464] REPLICATION: finished
0618 13:59:34.386560 (+ 833us) write_op.cc:301] APPLY: starting
0618 13:59:34.386585 (+ 25us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:34.386648 (+ 63us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:34.386684 (+ 36us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:34.386776 (+ 92us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:34.386837 (+ 61us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:34.386849 (+ 12us) write_op.cc:312] APPLY: finished
0618 13:59:34.386962 (+ 113us) log.cc:844] Serialized 29 byte log entry
0618 13:59:34.387040 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.392513 (+ 5473us) write_op.cc:454] Released schema lock
0618 13:59:34.392525 (+ 12us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:34.392641 (+ 116us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.392648 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":745,"apply.run_cpu_time_us":655,"apply.run_wall_time_us":6074,"num_ops":2,"prepare.queue_time_us":280,"prepare.run_cpu_time_us":677,"prepare.run_wall_time_us":1009,"replication_time_us":109224}]]}
I20260618 13:59:34.522608 29984 raft_consensus.cc:493] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.522780 29984 raft_consensus.cc:515] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.523391 29984 leader_election.cc:290] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.529740 29566 ts_tablet_manager.cc:1929] T 8fd9fa9675234db2abff694a517c0ea8 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:34.529841 29566 log.cc:1199] T 8fd9fa9675234db2abff694a517c0ea8 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/8fd9fa9675234db2abff694a517c0ea8
I20260618 13:59:34.530160 29566 ts_tablet_manager.cc:1950] T 8fd9fa9675234db2abff694a517c0ea8 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:34.533540 28798 tablet_bootstrap.cc:492] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:34.533699 28798 ts_tablet_manager.cc:1403] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.052s user 0.004s sys 0.000s
I20260618 13:59:34.534432 28798 raft_consensus.cc:359] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.534690 28798 raft_consensus.cc:385] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.534758 28798 raft_consensus.cc:740] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.535228 28798 consensus_queue.cc:260] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.535861 28798 ts_tablet_manager.cc:1434] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:34.543020 28798 tablet_bootstrap.cc:492] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:34.553997 28798 tablet_bootstrap.cc:654] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:34.555137 29026 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 3f5d4296952e48a48756e6f27140da87 consensus state has no leader
I20260618 13:59:34.556423 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet c84189a4b1194c87bbdf81f6e0ed56e4 with delete_type TABLET_DATA_DELETED (replaced by tablet be00bfa9bc1a4b83b18bfad05ba1495e) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.557147 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet b51d0851efb34eaa9e9e4be6b292ef62 with delete_type TABLET_DATA_DELETED (replaced by tablet f9f7359c4a144ca89fe3e51d1365a425) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.557982 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.558570 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 516421cd843b44769efa2b555697c065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.559122 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f5a334d18ba4c06bb358fa86d8af80d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.559710 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet d9c00003300246ada2a356e3e1cf684b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.560266 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet c4cfca9e76e641a7bc0b58b9ee240f36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.560822 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet f3ba9186b3ff47d5a2b158fef619686e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.561362 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet c3d523419ff244f085d6eac1a3231973 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.561916 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet fddea8fba4da40d7a5780e6750caf5e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.562551 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 37a6764a5e024903848561c9935b1a43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.564215 29640 raft_consensus.cc:493] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.564380 29640 raft_consensus.cc:515] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.565127 29640 leader_election.cc:290] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.565552 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92ddbdfdfff547588a03f453e413af8f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:34.566109 28618 leader_election.cc:343] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.566745 28544 raft_consensus.cc:3060] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.567031 29984 raft_consensus.cc:493] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.567191 29984 raft_consensus.cc:515] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.569492 29640 raft_consensus.cc:493] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.569636 29640 raft_consensus.cc:515] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.570327 29640 leader_election.cc:290] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.571475 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet b6dd422c05324c989e29b20ae2afabad with delete_type TABLET_DATA_DELETED (replaced by tablet 97a9f949999446f09359629fe483bf67) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.571553 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b95bdb0e36584d92b9c28de257553827" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:34.572024 28618 leader_election.cc:343] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.572515 28798 tablet_bootstrap.cc:492] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:34.572598 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 4c2b924a096b479cad31c29bf7c99172 with delete_type TABLET_DATA_DELETED (replaced by tablet ab521028ac1a4d0da013112d76bb169f) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.572768 28798 ts_tablet_manager.cc:1403] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.002s
I20260618 13:59:34.573400 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet dd2385e6752d49429422fc854539e9c6 with delete_type TABLET_DATA_DELETED (replaced by tablet c38a2156a55e4e5eb91d101e9a61eecd) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.574076 28798 raft_consensus.cc:359] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.574285 28798 raft_consensus.cc:385] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.574353 28798 raft_consensus.cc:740] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.574699 28798 consensus_queue.cc:260] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.575376 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 233040a16b0d4b61b977ca57c1afec2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.581859 28798 ts_tablet_manager.cc:1434] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.009s user 0.000s sys 0.002s
I20260618 13:59:34.584573 29745 ts_tablet_manager.cc:1950] T dd2385e6752d49429422fc854539e9c6 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:34.584969 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 54) took 1679 ms. Trace:
I20260618 13:59:34.585074 29745 rpcz_store.cc:276] 0618 13:59:32.905679 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.907093 (+ 1414us) service_pool.cc:224] Handling call
0618 13:59:34.228812 (+1321719us) ts_tablet_manager.cc:1151] Deleting tablet dd2385e6752d49429422fc854539e9c6
0618 13:59:34.228825 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:34.315803 (+ 86978us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:34.422160 (+106357us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:34.584950 (+162790us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1321555}
I20260618 13:59:34.585455 29745 tablet_replica.cc:333] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:34.585690 29745 raft_consensus.cc:2243] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:34.585863 29745 raft_consensus.cc:2272] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:34.440297 28346 leader_election.cc:343] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.586551 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 55) took 2551 ms. Trace:
W20260618 13:59:34.586618 28346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 146618us
I20260618 13:59:34.586629 29566 rpcz_store.cc:276] 0618 13:59:32.035183 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.035249 (+ 66us) service_pool.cc:224] Handling call
0618 13:59:34.428823 (+2393574us) ts_tablet_manager.cc:1151] Deleting tablet 8fd9fa9675234db2abff694a517c0ea8
0618 13:59:34.428834 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:34.526810 (+ 97976us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:34.529692 (+ 2882us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:34.586532 (+ 56840us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2393354}
I20260618 13:59:34.586977 29566 tablet_replica.cc:333] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:34.587175 29566 raft_consensus.cc:2243] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:34.587481 29566 raft_consensus.cc:2272] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:34.587891 29679 leader_election.cc:290] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.588011 29566 ts_tablet_manager.cc:1916] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:34.588153 29679 raft_consensus.cc:493] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.588280 29679 raft_consensus.cc:515] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.591120 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d046e0421142cd84c5b12d6a434f53" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.591250 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d046e0421142cd84c5b12d6a434f53" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.591797 29640 raft_consensus.cc:493] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.591935 29640 raft_consensus.cc:515] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.592482 29640 leader_election.cc:290] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:34.593750 29745 ts_tablet_manager.cc:1916] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:34.594869 28617 leader_election.cc:304] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:34.595242 29640 raft_consensus.cc:2804] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:34.595326 29640 raft_consensus.cc:493] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:34.595400 29640 raft_consensus.cc:3060] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.595458 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d3d363766546c4a97b0763bad11f71" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:34.596030 28618 leader_election.cc:343] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.597386 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2df9e9e811ec4c2ea68467b1294bb62e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:34.597559 28403 raft_consensus.cc:2468] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:34.598392 28399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92ddbdfdfff547588a03f453e413af8f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:34.598522 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a10e83d26b704201b95e7c0c44a1a4f5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:34.598593 28399 raft_consensus.cc:2468] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:34.598623 28411 raft_consensus.cc:3060] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.601814 28406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "015494374645469785f1f978eaf0e9a5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:34.602000 28406 raft_consensus.cc:2468] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:34.602239 28405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be00bfa9bc1a4b83b18bfad05ba1495e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:34.602362 28405 raft_consensus.cc:2468] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:34.604694 29984 leader_election.cc:290] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:34.605279 28405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b95bdb0e36584d92b9c28de257553827" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:34.605412 28405 raft_consensus.cc:2468] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:34.606120 29914 raft_consensus.cc:493] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.606233 29914 raft_consensus.cc:515] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.606366 28799 tablet_bootstrap.cc:492] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:34.606499 28799 ts_tablet_manager.cc:1403] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.234s user 0.029s sys 0.034s
I20260618 13:59:34.607573 29679 leader_election.cc:290] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.610203 29914 leader_election.cc:290] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:34.611573 28483 leader_election.cc:343] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: e025b64380314546af8a1ab90c817825
I20260618 13:59:34.611687 28483 leader_election.cc:304] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.611929 28548 raft_consensus.cc:3060] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:34.612008 28483 leader_election.cc:343] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 233040a16b0d4b61b977ca57c1afec2d
I20260618 13:59:34.611981 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet dd2385e6752d49429422fc854539e9c6 (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]) successfully deleted
I20260618 13:59:34.612097 28483 leader_election.cc:304] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.612195 29749 raft_consensus.cc:2749] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:34.612367 29749 raft_consensus.cc:2749] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:34.614602 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a0039a463ec4a9c82a21a3a69b565fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.615195 28617 leader_election.cc:304] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:34.616030 29303 raft_consensus.cc:2804] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:34.616118 29303 raft_consensus.cc:493] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:34.616170 29303 raft_consensus.cc:3060] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.617354 28617 leader_election.cc:304] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:34.620906 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 2188ebf1727240cd886ba7662f8fcf43 with delete_type TABLET_DATA_DELETED (replaced by tablet c0a42aa779ae452e9420d730bf8d7d55) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.621476 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet d4667fe48f0f4eff9c6c8baabdab58d4 with delete_type TABLET_DATA_DELETED (replaced by tablet a662bb37fb7d4df49e61e524808b67a8) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.612351 28405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d3d363766546c4a97b0763bad11f71" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:34.623098 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b88ed7cf79c4929a8704f66691cbed4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.623121 28405 raft_consensus.cc:2468] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:34.623441 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet e025b64380314546af8a1ab90c817825 with delete_type TABLET_DATA_DELETED (replaced by tablet ece7909d056c4ba8a5dca6423f0c1781) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.623634 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet d7a97176f72349cda221baaea61d18ad with delete_type TABLET_DATA_DELETED (replaced by tablet 72fd63e8245d4042b091593881ba11b6) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.625519 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet 897bf6d6bbee4ad89c296325d94a3e53 with delete_type TABLET_DATA_DELETED (replaced by tablet e196dd3c7fe14524a50152643ddbc2ad) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.625860 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet 316f470ea82f42fca48e4e96d49c9574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.626178 29714 raft_consensus.cc:2804] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:34.626186 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet d1baf86497d442dbbbddbc3498bebbee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.626256 29714 raft_consensus.cc:493] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:34.626309 29714 raft_consensus.cc:3060] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.626507 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet 18d8b609b96a4e81b57f69bff522eeea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.626828 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet 69d5e75b4a8247b29d69d07fad5ca5db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.627148 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet 7aa9f65d8a5c46fdb745ee17af946dce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.627457 30008 consensus_queue.cc:1048] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:34.627612 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a51e830e3014726b1d1c2608973f036 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.627810 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b59255b14df4158afe37cb8890e7415 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.628315 28411 raft_consensus.cc:2468] T a10e83d26b704201b95e7c0c44a1a4f5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:34.628422 29746 ts_tablet_manager.cc:1929] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:34.628515 29746 log.cc:1199] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/da4d4f03911f4ac099ecc2fa1d8af494
I20260618 13:59:34.628835 29746 ts_tablet_manager.cc:1950] T da4d4f03911f4ac099ecc2fa1d8af494 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:34.629009 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet e3d8616846c649b6a969b971db0d03db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:34.629271 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet f89c9954d48e407c96c294c26644e25c with delete_type TABLET_DATA_DELETED (replaced by tablet b0d3d363766546c4a97b0763bad11f71) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:34.629526 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet f278d0acbb0c4a0ca61640bc142e6335 with delete_type TABLET_DATA_DELETED (replaced by tablet 72c04f55890648f0a8184f3783243f7c) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:34.629978 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 35d8dd2c388e43a0be380aa953ec2c82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:34 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:34.630250 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 316f470ea82f42fca48e4e96d49c9574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:34.630676 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet d3dbf13544ce408ab97834aec5862390 with delete_type TABLET_DATA_DELETED (replaced by tablet ba18e879631947f88bd266681746ec82) from {username='slave'} at 127.0.0.1:38230
W20260618 13:59:34.631429 28346 leader_election.cc:343] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.631719 28346 leader_election.cc:304] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:34.634081 28346 leader_election.cc:343] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.634281 28346 leader_election.cc:304] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.634613 29984 raft_consensus.cc:2749] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:34.635079 28346 leader_election.cc:343] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.635253 28346 leader_election.cc:304] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.635663 28386 tablet_service.cc:1558] Processing DeleteTablet for tablet fddea8fba4da40d7a5780e6750caf5e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:34.635910 30022 raft_consensus.cc:2749] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:34.636973 29640 raft_consensus.cc:515] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.637501 30022 raft_consensus.cc:2749] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:34.637595 28346 leader_election.cc:343] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.637658 29640 leader_election.cc:290] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.638121 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591465e2b57247a2b224cc8b09225811" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.638207 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591465e2b57247a2b224cc8b09225811" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.638296 28680 raft_consensus.cc:2468] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:34.639000 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "438663f365e34fab87e34f434920cecf" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.639917 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450c68da26404767b01c5e8da3f6286d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:34.640275 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55ae29c3b7ed44cfa659130745037aa6" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:34.640345 28618 leader_election.cc:343] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.640413 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55ae29c3b7ed44cfa659130745037aa6" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.640832 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "438663f365e34fab87e34f434920cecf" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.641893 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b509f940a02545019c526e45e8372c97" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.641999 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet 36b8a46a41e543c58a946b6764618c7d with delete_type TABLET_DATA_DELETED (replaced by tablet f89c9954d48e407c96c294c26644e25c) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.642525 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c75278db2234b11bf95f1cf5cc76edd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.642746 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b509f940a02545019c526e45e8372c97" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.643081 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet d6871232860b41d489e34d95eb897241 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.643369 29984 raft_consensus.cc:493] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.643468 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet 4a01dfb4d59549399c94640f16045555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:57282
W20260618 13:59:34.643553 28346 consensus_peers.cc:597] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:34.643512 29984 raft_consensus.cc:515] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.643843 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet ba0d05ea44e1473fb4b24f61eab096e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:57282
W20260618 13:59:34.643822 28346 leader_election.cc:343] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.643962 28346 leader_election.cc:304] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.644059 29984 leader_election.cc:290] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.644222 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet eead369de58f40378254697cbbbc6958 with delete_type TABLET_DATA_DELETED (replaced by tablet d464dd402d2044469e364ed37fdfb39b) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.644243 29984 raft_consensus.cc:2749] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:34.645398 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet edc72a06ee8e4420b07a17d2feb3fcc2 with delete_type TABLET_DATA_DELETED (replaced by tablet aa8cf927bed9437086efd8e0f21ba26a) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.645516 28549 raft_consensus.cc:3060] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.645488 28799 raft_consensus.cc:359] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.645677 28799 raft_consensus.cc:385] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.645694 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet 94365c44b5e24acab5df60c67203794a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.645735 28799 raft_consensus.cc:740] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.645962 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet f89c9954d48e407c96c294c26644e25c with delete_type TABLET_DATA_DELETED (replaced by tablet b0d3d363766546c4a97b0763bad11f71) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.645941 28799 consensus_queue.cc:260] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.646231 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 8fd9fa9675234db2abff694a517c0ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet 538077e459a44ed78c95f2e881b6e0b0) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.646390 28799 ts_tablet_manager.cc:1434] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.040s user 0.002s sys 0.000s
I20260618 13:59:34.646409 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet 41107eb1d9164a9c98060003b4027412 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f5d4296952e48a48756e6f27140da87) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.646677 28799 tablet_bootstrap.cc:492] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:34.646867 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet fca76774f119435899188313a33dad98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.647224 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet dca54111fea047d9ae0fabe376d89382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.648290 28799 tablet_bootstrap.cc:654] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.652679 29566 ts_tablet_manager.cc:1929] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:34.652792 29566 log.cc:1199] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/092a9aba9c6e48f5be9cf5d55ba93678
I20260618 13:59:34.653163 29566 ts_tablet_manager.cc:1950] T 092a9aba9c6e48f5be9cf5d55ba93678 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:34.653414 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet aef7c1f111634c98af046e73fa9ffdeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.653864 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet 75824a5d2f7e47838b60e6a2b785c6b2 with delete_type TABLET_DATA_DELETED (replaced by tablet d4539f767c054e9e8ebc1f3f56108e7e) from {username='slave'} at 127.0.0.1:57282
W20260618 13:59:34.655586 29026 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.1:48495 (request call id 143) took 2677 ms (client timeout 1000 ms). Trace:
I20260618 13:59:34.655838 28617 leader_election.cc:304] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:34.656168 29719 raft_consensus.cc:2804] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:34.656236 29719 raft_consensus.cc:493] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:34.656296 29719 raft_consensus.cc:3060] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:34.656404 28617 leader_election.cc:343] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: c4cfca9e76e641a7bc0b58b9ee240f36
I20260618 13:59:34.656527 28617 leader_election.cc:304] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:34.656767 30008 raft_consensus.cc:2749] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:34.657505 28617 leader_election.cc:304] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:34.657750 28799 tablet_bootstrap.cc:492] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:34.657840 30008 raft_consensus.cc:2804] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:34.657932 30008 raft_consensus.cc:493] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:34.658033 30008 raft_consensus.cc:3060] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.658037 28799 ts_tablet_manager.cc:1403] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.007s
I20260618 13:59:34.658505 28617 leader_election.cc:304] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:34.658928 28617 leader_election.cc:304] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:34.658957 29400 raft_consensus.cc:2804] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:34.659061 29400 raft_consensus.cc:493] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:34.659185 29400 raft_consensus.cc:3060] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.659838 28799 raft_consensus.cc:359] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.660091 28799 raft_consensus.cc:385] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.660120 29640 raft_consensus.cc:2804] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:34.660197 28799 raft_consensus.cc:740] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.660290 29640 raft_consensus.cc:493] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:34.660460 29640 raft_consensus.cc:3060] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.660544 28799 consensus_queue.cc:260] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:34.656891 29026 rpcz_store.cc:269] 0618 13:59:31.978453 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:31.979905 (+ 1452us) service_pool.cc:224] Handling call
0618 13:59:32.088145 (+108240us) spinlock_profiling.cc:243] Waited 73.1 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb467ef08 00007fcbab0ddca6 00007fcbab0dd9d5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:34.655537 (+2567392us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":226142,"reported_tablets":169,"spinlock_wait_cycles":168221440}
I20260618 13:59:34.661309 28799 ts_tablet_manager.cc:1434] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:34.661691 28799 tablet_bootstrap.cc:492] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:34.662338 29026 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_11b11bd80db34f639f805447f8027a27"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:34.663502 28799 tablet_bootstrap.cc:654] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.666775 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet 092a9aba9c6e48f5be9cf5d55ba93678 with delete_type TABLET_DATA_DELETED (replaced by tablet c7f42997416841a7b2c1ad347d2e9282) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.667317 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet d502d67826ff4d7a8b0f515e5d8b6d77 with delete_type TABLET_DATA_DELETED (replaced by tablet 657e55fa53c447babfe91b1582806e6b) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.667719 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet fe750fc5d998448caeab0a6c808dfba7 with delete_type TABLET_DATA_DELETED (replaced by tablet 438663f365e34fab87e34f434920cecf) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.669031 28798 tablet_bootstrap.cc:492] T 75824a5d2f7e47838b60e6a2b785c6b2 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:34.669683 28797 tablet_bootstrap.cc:492] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:34.672506 28798 tablet_bootstrap.cc:654] T 75824a5d2f7e47838b60e6a2b785c6b2 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:34.672761 29963 log.cc:927] Time spent T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Append to log took a long time: real 0.090s user 0.000s sys 0.000s
I20260618 13:59:34.673287 28797 ts_tablet_manager.cc:1403] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.309s user 0.022s sys 0.059s
I20260618 13:59:34.674942 28797 raft_consensus.cc:359] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:34.644726 28346 leader_election.cc:343] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:34.678821 28346 leader_election.cc:343] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:34.679145 28346 leader_election.cc:343] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:34.679673 28346 leader_election.cc:343] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.680330 28797 raft_consensus.cc:385] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.680943 28797 raft_consensus.cc:740] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.681643 28797 consensus_queue.cc:260] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:34.685446 28539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0463fa2e30554abcbfc8f37b702e81d4" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.697639 28346 leader_election.cc:304] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:34.697971 29914 raft_consensus.cc:2804] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:34.698046 29914 raft_consensus.cc:493] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:34.698107 29914 raft_consensus.cc:3060] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:34.698120 28346 leader_election.cc:343] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.698205 28346 leader_election.cc:304] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.698544 30022 raft_consensus.cc:2749] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:34.698652 29984 consensus_queue.cc:1048] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:34.699332 28346 leader_election.cc:343] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.699556 28346 leader_election.cc:304] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.699884 29984 raft_consensus.cc:2749] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:34.700495 28346 leader_election.cc:343] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.700711 28346 leader_election.cc:304] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.701072 29984 raft_consensus.cc:2749] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:34.701926 28346 leader_election.cc:343] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.705250 28797 ts_tablet_manager.cc:1434] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.029s user 0.003s sys 0.003s
I20260618 13:59:34.707389 29984 raft_consensus.cc:493] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.707561 29984 raft_consensus.cc:515] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:34.708297 29984 leader_election.cc:290] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:34.711361 29026 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11b11bd80db34f639f805447f8027a27 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:34.713442 28797 tablet_bootstrap.cc:492] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:34.714293 29025 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_5159a242b12044fc9a0b3e535ea40050 [id=cfb4ca8c4e884ccc8602be52d0b39100] alter complete (version 4)
I20260618 13:59:34.717372 28797 tablet_bootstrap.cc:654] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.717708 30032 tablet.cc:1724] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:34.720760 29759 raft_consensus.cc:493] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:34.720943 29759 raft_consensus.cc:515] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.721686 29759 leader_election.cc:290] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:34.726121 28483 leader_election.cc:343] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 8b88ed7cf79c4929a8704f66691cbed4
W20260618 13:59:34.718564 28617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 59529.5us
I20260618 13:59:34.726977 28617 leader_election.cc:304] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:34.728307 29745 ts_tablet_manager.cc:1929] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:34.728531 29745 log.cc:1199] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/283523e5b22843ba935eabe6283d3a99
I20260618 13:59:34.728811 30019 tablet.cc:1724] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:34.729031 29745 ts_tablet_manager.cc:1950] T 283523e5b22843ba935eabe6283d3a99 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:34.729853 29914 raft_consensus.cc:515] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.730595 29914 leader_election.cc:290] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.731392 29746 tablet_replica.cc:333] T aa8cf927bed9437086efd8e0f21ba26a P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:34.731616 29746 raft_consensus.cc:2243] T aa8cf927bed9437086efd8e0f21ba26a P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:34.731769 29746 raft_consensus.cc:2272] T aa8cf927bed9437086efd8e0f21ba26a P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:34.732371 29746 ts_tablet_manager.cc:1916] T aa8cf927bed9437086efd8e0f21ba26a P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:34.732496 28799 tablet_bootstrap.cc:492] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:34.732612 28799 ts_tablet_manager.cc:1403] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.071s user 0.006s sys 0.009s
I20260618 13:59:34.733634 28799 raft_consensus.cc:359] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.733788 28799 raft_consensus.cc:385] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.733839 28799 raft_consensus.cc:740] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.734084 28799 consensus_queue.cc:260] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.734788 28799 ts_tablet_manager.cc:1434] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:34.735028 28799 tablet_bootstrap.cc:492] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:34.738204 29914 raft_consensus.cc:493] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.738351 29914 raft_consensus.cc:515] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.739125 29914 leader_election.cc:290] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.741425 29303 raft_consensus.cc:515] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.742372 29303 leader_election.cc:290] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.744290 28798 tablet_bootstrap.cc:492] T 75824a5d2f7e47838b60e6a2b785c6b2 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:34.744481 28798 ts_tablet_manager.cc:1403] T 75824a5d2f7e47838b60e6a2b785c6b2 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.076s user 0.005s sys 0.000s
I20260618 13:59:34.744817 30034 raft_consensus.cc:2804] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:34.745436 30034 raft_consensus.cc:697] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:34.746490 28798 raft_consensus.cc:359] T 75824a5d2f7e47838b60e6a2b785c6b2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.746737 28798 raft_consensus.cc:385] T 75824a5d2f7e47838b60e6a2b785c6b2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.746908 28798 raft_consensus.cc:740] T 75824a5d2f7e47838b60e6a2b785c6b2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.747215 28798 consensus_queue.cc:260] T 75824a5d2f7e47838b60e6a2b785c6b2 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.747887 28798 ts_tablet_manager.cc:1434] T 75824a5d2f7e47838b60e6a2b785c6b2 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:34.748142 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 56) took 2700 ms. Trace:
I20260618 13:59:34.748210 28798 tablet_bootstrap.cc:492] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:34.748260 29566 rpcz_store.cc:276] 0618 13:59:32.042503 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.042575 (+ 72us) service_pool.cc:224] Handling call
0618 13:59:34.586931 (+2544356us) ts_tablet_manager.cc:1151] Deleting tablet 092a9aba9c6e48f5be9cf5d55ba93678
0618 13:59:34.586942 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:34.629562 (+ 42620us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:34.652612 (+ 23050us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:34.742896 (+ 90284us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2544174}
I20260618 13:59:34.748678 29566 tablet_replica.cc:333] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:34.748895 29566 raft_consensus.cc:2243] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:34.749249 29566 raft_consensus.cc:2272] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:34.749843 29566 ts_tablet_manager.cc:1916] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:34.751679 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450c68da26404767b01c5e8da3f6286d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:34.751873 28411 raft_consensus.cc:3060] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.752622 29303 raft_consensus.cc:493] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.752769 29303 raft_consensus.cc:515] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.753448 29303 leader_election.cc:290] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.757670 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2df9e9e811ec4c2ea68467b1294bb62e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:34.757875 28409 raft_consensus.cc:3060] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.758952 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0463fa2e30554abcbfc8f37b702e81d4" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.759153 28680 raft_consensus.cc:2468] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
W20260618 13:59:34.758287 28618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 51421.3us
I20260618 13:59:34.759713 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2df9e9e811ec4c2ea68467b1294bb62e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:34.761487 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d523419ff244f085d6eac1a3231973" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.763710 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d523419ff244f085d6eac1a3231973" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:34.763897 28404 raft_consensus.cc:2468] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:34.753638 30034 consensus_queue.cc:237] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.765398 29914 raft_consensus.cc:493] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.765355 30029 consensus_queue.cc:1048] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260618 13:59:34.765516 29914 raft_consensus.cc:515] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.766088 29914 leader_election.cc:290] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.766528 28797 tablet_bootstrap.cc:492] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:34.766772 28797 ts_tablet_manager.cc:1403] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.053s user 0.005s sys 0.001s
W20260618 13:59:34.767802 28618 leader_election.cc:343] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.768036 29303 consensus_queue.cc:1048] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:34.768138 28618 leader_election.cc:343] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:34.768944 28484 leader_election.cc:343] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 8b88ed7cf79c4929a8704f66691cbed4
I20260618 13:59:34.769044 28484 leader_election.cc:304] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.769389 30032 tablet.cc:1724] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:34.769737 29759 raft_consensus.cc:2749] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:34.770226 29017 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 9c17e76ae9b44207998a43e3576908c4
I20260618 13:59:34.770687 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 55) took 1864 ms. Trace:
I20260618 13:59:34.770830 29745 rpcz_store.cc:276] 0618 13:59:32.905766 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.907292 (+ 1526us) service_pool.cc:224] Handling call
0618 13:59:34.585404 (+1678112us) ts_tablet_manager.cc:1151] Deleting tablet 283523e5b22843ba935eabe6283d3a99
0618 13:59:34.585413 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:34.629962 (+ 44549us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:34.728215 (+ 98253us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:34.770672 (+ 42457us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1677958}
I20260618 13:59:34.771239 29745 tablet_replica.cc:333] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:34.776002 28799 tablet_bootstrap.cc:654] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.777948 29303 consensus_queue.cc:1048] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:34.779727 28618 consensus_peers.cc:597] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:34.780283 28617 leader_election.cc:304] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:34.781095 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 8fd9fa9675234db2abff694a517c0ea8 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]) successfully deleted
W20260618 13:59:34.781435 28989 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.100.3:35503, user_credentials={real_user=slave}} blocked reactor thread for 53791.6us
I20260618 13:59:34.781905 30029 raft_consensus.cc:2804] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:34.782063 30029 raft_consensus.cc:493] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:34.783394 30029 raft_consensus.cc:3060] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.771368 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 283523e5b22843ba935eabe6283d3a99 (table default.Table_d16c53f18154437e8d06e88da66faf13 [id=cf2c2a972733485a9ba1d6f1e99937a1]) successfully deleted
I20260618 13:59:34.785514 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 9c17e76ae9b44207998a43e3576908c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 2128dbec33db453ba51070767a15b21e) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:34.796295 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "538077e459a44ed78c95f2e881b6e0b0" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.798270 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591465e2b57247a2b224cc8b09225811" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:34.798491 28682 raft_consensus.cc:3060] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.799379 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa24afe9db46408cace50f642f879f4b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.800061 30018 tablet.cc:1724] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:34.801079 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7766014ee9814f01ae7ac3e849ca3d50" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:34.802595 28617 consensus_peers.cc:597] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f89c9954d48e407c96c294c26644e25c. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:34.806170 29017 catalog_manager.cc:6784] ReplaceTablet: tablet 9c17e76ae9b44207998a43e3576908c4 deleted and replaced by tablet 2128dbec33db453ba51070767a15b21e
I20260618 13:59:34.807753 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 092a9aba9c6e48f5be9cf5d55ba93678 (table default.Table_0a13b90182654346acbbc8f47a56436d [id=c31d65adb55f458c9ced14d598464f89]) successfully deleted
I20260618 13:59:34.810683 29914 raft_consensus.cc:493] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.810833 29914 raft_consensus.cc:515] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.811317 28799 tablet_bootstrap.cc:492] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:34.811421 29914 leader_election.cc:290] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.811446 28799 ts_tablet_manager.cc:1403] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.076s user 0.005s sys 0.013s
I20260618 13:59:34.812402 28799 raft_consensus.cc:359] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:34.815649 29017 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 87) took 1602 ms. Trace:
I20260618 13:59:34.801678 28346 leader_election.cc:304] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
W20260618 13:59:34.823057 28346 leader_election.cc:343] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:34.823348 28346 leader_election.cc:343] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:34.823529 28346 leader_election.cc:343] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.823699 30022 raft_consensus.cc:2804] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:34.823767 30022 raft_consensus.cc:493] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:34.823840 30022 raft_consensus.cc:3060] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.825862 28799 raft_consensus.cc:385] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.826817 29017 rpcz_store.cc:276] 0618 13:59:33.212881 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.441491 (+1228610us) service_pool.cc:224] Handling call
0618 13:59:34.815606 (+374115us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:34.445908 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:34.446010 (+ 102us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:34.446016 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:34.446018 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:34.446113 (+ 95us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:34.446125 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:34.446137 (+ 12us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:34.446225 (+ 88us) tablet.cc:701] Row locks acquired
0618 13:59:34.446227 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:34.446258 (+ 31us) write_op.cc:270] Start()
0618 13:59:34.446286 (+ 28us) write_op.cc:276] Timestamp: P: 1781791174446221 usec, L: 0
0618 13:59:34.446289 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:34.446481 (+ 192us) log.cc:844] Serialized 659 byte log entry
0618 13:59:34.535628 (+ 89147us) op_driver.cc:464] REPLICATION: finished
0618 13:59:34.535737 (+ 109us) write_op.cc:301] APPLY: starting
0618 13:59:34.535751 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:34.535812 (+ 61us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:34.554181 (+ 18369us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:34.554325 (+ 144us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:34.554401 (+ 76us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:34.554443 (+ 42us) write_op.cc:312] APPLY: finished
0618 13:59:34.554560 (+ 117us) log.cc:844] Serialized 29 byte log entry
0618 13:59:34.554646 (+ 86us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.554674 (+ 28us) write_op.cc:454] Released schema lock
0618 13:59:34.554686 (+ 12us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:34.554791 (+ 105us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.554798 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":59,"apply.run_cpu_time_us":703,"apply.run_wall_time_us":19021,"num_ops":2,"prepare.queue_time_us":53,"prepare.run_cpu_time_us":656,"prepare.run_wall_time_us":656,"replication_time_us":89305}]]}
I20260618 13:59:34.828295 28799 raft_consensus.cc:740] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.828735 28799 consensus_queue.cc:260] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:34.828943 29714 raft_consensus.cc:515] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.829563 29714 leader_election.cc:290] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.829602 28799 ts_tablet_manager.cc:1434] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260618 13:59:34.829846 28411 raft_consensus.cc:2468] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:34.829928 28799 tablet_bootstrap.cc:492] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:34.839717 29914 raft_consensus.cc:493] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.839895 29914 raft_consensus.cc:515] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.840798 29914 leader_election.cc:290] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.842849 28617 leader_election.cc:304] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:34.843355 29714 raft_consensus.cc:2804] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:34.843358 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92ddbdfdfff547588a03f453e413af8f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:34.843485 29714 raft_consensus.cc:697] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:34.843544 28411 raft_consensus.cc:3060] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.843773 29714 consensus_queue.cc:237] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.844758 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad045effb32948b5b8e9be5bb7996577" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.854486 29745 raft_consensus.cc:2243] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:34.854892 29745 pending_rounds.cc:70] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b: Trying to abort 1 pending ops.
I20260618 13:59:34.854938 30032 tablet.cc:1724] T c881dd3cb66344969310956ad6e83cc0 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:34.854941 29745 pending_rounds.cc:77] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298216646828761088 op_type: NO_OP noop_request { }
I20260618 13:59:34.855110 29745 raft_consensus.cc:2889] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:34.855183 29745 raft_consensus.cc:2272] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:34.862807 28799 tablet_bootstrap.cc:654] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.862926 29745 ts_tablet_manager.cc:1916] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:34.867681 30031 consensus_queue.cc:1048] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:34.885504 29023 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.2:52853 (request call id 116) took 3481 ms (client timeout 1000 ms). Trace:
I20260618 13:59:34.885725 29749 raft_consensus.cc:493] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.885870 29749 raft_consensus.cc:515] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.886180 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "538077e459a44ed78c95f2e881b6e0b0" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.886701 28542 raft_consensus.cc:3060] T 75824a5d2f7e47838b60e6a2b785c6b2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:34.888680 28346 leader_election.cc:343] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.888679 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "957670718c8c4b8cb87115cc4a2fa861" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.888794 28346 leader_election.cc:304] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.889016 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa24afe9db46408cace50f642f879f4b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.889055 29914 raft_consensus.cc:2749] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:34.889333 28545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad045effb32948b5b8e9be5bb7996577" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:34.889662 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7766014ee9814f01ae7ac3e849ca3d50" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:34.891149 28346 leader_election.cc:343] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:34.891309 28346 leader_election.cc:343] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.891398 28346 leader_election.cc:304] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.891598 29679 raft_consensus.cc:2749] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:34.891731 28346 leader_election.cc:343] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:34.891904 28346 leader_election.cc:343] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.891987 28346 leader_election.cc:304] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.892166 29679 raft_consensus.cc:2749] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:34.896270 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "957670718c8c4b8cb87115cc4a2fa861" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.896664 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet b16f7d3c7837498e996bba1818110642 with delete_type TABLET_DATA_DELETED (replaced by tablet d1baf86497d442dbbbddbc3498bebbee) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.896756 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet bc0adedecee4405e86b0a37b9e62e352 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:57282
W20260618 13:59:34.896900 28346 leader_election.cc:343] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.896999 28346 leader_election.cc:304] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.897189 29679 raft_consensus.cc:2749] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:34.898231 28346 leader_election.cc:343] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.898332 28346 leader_election.cc:304] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:34.898535 29679 raft_consensus.cc:2749] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:34.896173 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591465e2b57247a2b224cc8b09225811" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:34.887018 29017 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_642894fa43114390bed040b60b648d86"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:34.899556 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet 751cd9bd961b4f56a588ed02a05f0212 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.900050 30050 tablet.cc:1724] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:34.901271 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92ddbdfdfff547588a03f453e413af8f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:34.902251 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet e95c88e4b0a046a3ad69ee49af47c7c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.903921 29759 raft_consensus.cc:493] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.904070 29759 raft_consensus.cc:515] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.909403 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd06bb7cffc34412b420d70435fce373" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:34.910506 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet bce7bf74b6d24d0bb43808b8a62cf1b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.911302 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet b0285c75c632457383e81805ab036aaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.912417 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 97a9f949999446f09359629fe483bf67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:29 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.913163 28797 raft_consensus.cc:359] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:34.914988 28346 leader_election.cc:343] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:34.915218 29017 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_642894fa43114390bed040b60b648d86 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:34.927397 28409 raft_consensus.cc:2468] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:34.928045 28411 raft_consensus.cc:2468] T 92ddbdfdfff547588a03f453e413af8f P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:34.928045 28799 tablet_bootstrap.cc:492] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:34.932135 28799 ts_tablet_manager.cc:1403] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.102s user 0.004s sys 0.004s
I20260618 13:59:34.932564 29759 leader_election.cc:290] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:34.902599 29749 leader_election.cc:290] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:34.933467 28799 raft_consensus.cc:359] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:34.938478 28799 raft_consensus.cc:385] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.942483 28799 raft_consensus.cc:740] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:34.943027 28799 consensus_queue.cc:260] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:34.943871 28799 ts_tablet_manager.cc:1434] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260618 13:59:34.945799 28799 tablet_bootstrap.cc:492] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:34.918242 28617 consensus_peers.cc:597] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:34.946205 28617 leader_election.cc:304] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:34.946666 29721 raft_consensus.cc:2804] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:34.946830 29721 raft_consensus.cc:697] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:34.947130 29721 consensus_queue.cc:237] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.922501 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd06bb7cffc34412b420d70435fce373" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:34.923116 28618 leader_election.cc:343] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:34.950578 28618 consensus_peers.cc:597] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:34.926038 30051 tablet.cc:1724] T c881dd3cb66344969310956ad6e83cc0 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:34.950800 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b51d0851efb34eaa9e9e4be6b292ef62" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:34.951234 28799 tablet_bootstrap.cc:654] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.927806 29566 ts_tablet_manager.cc:1929] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:34.951515 30030 raft_consensus.cc:493] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.951567 29566 log.cc:1199] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/d6871232860b41d489e34d95eb897241
I20260618 13:59:34.951646 30030 raft_consensus.cc:515] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.931196 29400 raft_consensus.cc:515] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.927397 30008 raft_consensus.cc:515] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:34.953032 29566 ts_tablet_manager.cc:1950] T d6871232860b41d489e34d95eb897241 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:34.953739 28484 leader_election.cc:304] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:34.955943 29023 rpcz_store.cc:269] 0618 13:59:31.404324 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:31.648081 (+243757us) service_pool.cc:224] Handling call
0618 13:59:34.885435 (+3237354us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:32.204394 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:32.204538 (+ 144us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:32.204543 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:32.204545 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:32.204826 (+ 281us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:32.204841 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:32.204846 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:32.204912 (+ 66us) tablet.cc:701] Row locks acquired
0618 13:59:32.204914 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:32.204945 (+ 31us) write_op.cc:270] Start()
0618 13:59:32.204979 (+ 34us) write_op.cc:276] Timestamp: P: 1781791172204925 usec, L: 0
0618 13:59:32.204982 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:32.205174 (+ 192us) log.cc:844] Serialized 356 byte log entry
0618 13:59:32.209911 (+ 4737us) op_driver.cc:464] REPLICATION: finished
0618 13:59:32.210977 (+ 1066us) write_op.cc:301] APPLY: starting
0618 13:59:32.210995 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:32.211039 (+ 44us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:32.211098 (+ 59us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:32.211166 (+ 68us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:32.211200 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:32.211207 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:32.211297 (+ 90us) log.cc:844] Serialized 23 byte log entry
0618 13:59:32.211538 (+ 241us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.211556 (+ 18us) write_op.cc:454] Released schema lock
0618 13:59:32.211564 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:32.211755 (+ 191us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:32.211762 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":386242,"reported_tablets":183,"child_traces":[["op",{"apply.queue_time_us":682,"apply.run_cpu_time_us":502,"apply.run_wall_time_us":768,"num_ops":1,"prepare.queue_time_us":144329,"prepare.run_cpu_time_us":778,"prepare.run_wall_time_us":1660,"raft.queue_time_us":63,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":235,"replication_time_us":4899}]]}
I20260618 13:59:34.957403 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b51d0851efb34eaa9e9e4be6b292ef62" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:34.957801 29749 raft_consensus.cc:2749] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:34.958354 29746 ts_tablet_manager.cc:1929] T aa8cf927bed9437086efd8e0f21ba26a P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:34.958504 29746 log.cc:1199] T aa8cf927bed9437086efd8e0f21ba26a P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/aa8cf927bed9437086efd8e0f21ba26a
I20260618 13:59:34.958930 29746 ts_tablet_manager.cc:1950] T aa8cf927bed9437086efd8e0f21ba26a P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:34.960809 30008 leader_election.cc:290] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.961622 29400 leader_election.cc:290] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.961768 28617 leader_election.cc:304] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:34.962040 29400 raft_consensus.cc:2804] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:34.963452 29400 raft_consensus.cc:697] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:34.963624 30030 leader_election.cc:290] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.963786 29400 consensus_queue.cc:237] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:34.965286 29023 master_service.cc:960] ReplaceTablet: received request to replace tablet ab521028ac1a4d0da013112d76bb169f from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:34.966588 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be00bfa9bc1a4b83b18bfad05ba1495e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:34.966761 28409 raft_consensus.cc:3060] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.967540 30032 tablet.cc:1724] T 15180aefba0544329daee74308ef5010 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:34.968943 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b95bdb0e36584d92b9c28de257553827" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:34.969076 28411 raft_consensus.cc:3060] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:34.970080 30008 consensus_queue.cc:1048] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:34.971941 29024 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.2:52853 (request call id 115) took 4572 ms (client timeout 1000 ms). Trace:
I20260618 13:59:34.972635 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572d17724d249739be1a1cb9468f909" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:34.973083 28403 raft_consensus.cc:2468] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:34.973654 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 9c17e76ae9b44207998a43e3576908c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 2128dbec33db453ba51070767a15b21e) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:34.973213 29024 rpcz_store.cc:269] 0618 13:59:30.349684 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:30.362142 (+ 12458us) service_pool.cc:224] Handling call
0618 13:59:30.388947 (+ 26805us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb467ef08 00007fcbab0ddca6 00007fcbab0dd9d5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:30.973939 (+584992us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:30.973947 (+ 8us) write_op.cc:454] Released schema lock
0618 13:59:34.922227 (+3948280us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:30.927692 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:30.927807 (+ 115us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:30.927814 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:30.927816 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:30.927925 (+ 109us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:30.927938 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:30.927944 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:30.928066 (+ 122us) tablet.cc:701] Row locks acquired
0618 13:59:30.928068 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:30.928100 (+ 32us) write_op.cc:270] Start()
0618 13:59:30.928142 (+ 42us) write_op.cc:276] Timestamp: P: 1781791170928047 usec, L: 0
0618 13:59:30.928145 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:30.928346 (+ 201us) log.cc:844] Serialized 907 byte log entry
0618 13:59:30.934722 (+ 6376us) op_driver.cc:464] REPLICATION: finished
0618 13:59:30.934860 (+ 138us) write_op.cc:301] APPLY: starting
0618 13:59:30.934876 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:30.934923 (+ 47us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:30.934960 (+ 37us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:30.935027 (+ 67us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:30.935075 (+ 48us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:30.935082 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:30.935173 (+ 91us) log.cc:844] Serialized 29 byte log entry
0618 13:59:30.935240 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:30.935262 (+ 22us) write_op.cc:454] Released schema lock
0618 13:59:30.935269 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":18884,"reported_tablets":179,"spinlock_wait_cycles":54424960,"child_traces":[["op",{"apply.queue_time_us":89,"apply.run_cpu_time_us":465,"apply.run_wall_time_us":464,"num_ops":2,"prepare.queue_time_us":48,"prepare.run_cpu_time_us":776,"prepare.run_wall_time_us":775,"raft.queue_time_us":809,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":214,"replication_time_us":6544}]]}
I20260618 13:59:34.974545 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet e5f7eaa7d218440ca2c35340c5c87814 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.972962 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet da4d4f03911f4ac099ecc2fa1d8af494 (table default.Table_99428babdde649b8b5d5fab801b58136 [id=f7cb92d4ab0742af9946f6bcf708df4d]) successfully deleted
I20260618 13:59:34.979115 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet ca2ee746c5024cfc9b31c470f9616e7d with delete_type TABLET_DATA_DELETED (replaced by tablet 60b6216474b74a14af6abf977e312b77) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.979544 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 3dd0eaa64271489d92aa380df877f257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.979970 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 316f470ea82f42fca48e4e96d49c9574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.980345 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 2188ebf1727240cd886ba7662f8fcf43 with delete_type TABLET_DATA_DELETED (replaced by tablet c0a42aa779ae452e9420d730bf8d7d55) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.980603 29024 master_service.cc:960] ReplaceTablet: received request to replace tablet f45556a59c424ca2bc30956566374c89 from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:34.980935 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a0039a463ec4a9c82a21a3a69b565fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.981667 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet 8fd9fa9675234db2abff694a517c0ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet 538077e459a44ed78c95f2e881b6e0b0) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.982069 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet 092a9aba9c6e48f5be9cf5d55ba93678 with delete_type TABLET_DATA_DELETED (replaced by tablet c7f42997416841a7b2c1ad347d2e9282) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.982738 30061 tablet.cc:1724] T 15180aefba0544329daee74308ef5010 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:34.983536 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet 41107eb1d9164a9c98060003b4027412 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f5d4296952e48a48756e6f27140da87) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.983968 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet aef7c1f111634c98af046e73fa9ffdeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.984375 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet 233040a16b0d4b61b977ca57c1afec2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.984737 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b88ed7cf79c4929a8704f66691cbed4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.985124 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet b6dd422c05324c989e29b20ae2afabad with delete_type TABLET_DATA_DELETED (replaced by tablet 97a9f949999446f09359629fe483bf67) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.985467 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet d9c00003300246ada2a356e3e1cf684b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.986040 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet c4cfca9e76e641a7bc0b58b9ee240f36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.986878 29984 raft_consensus.cc:493] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:34.987040 29984 raft_consensus.cc:515] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.987644 29984 leader_election.cc:290] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:34.988361 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet 897bf6d6bbee4ad89c296325d94a3e53 with delete_type TABLET_DATA_DELETED (replaced by tablet e196dd3c7fe14524a50152643ddbc2ad) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:34.989091 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9024952ab8094fcda1329562bbde8ef5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:34.989564 28346 leader_election.cc:343] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:34.991010 28798 tablet_bootstrap.cc:654] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:34.994764 28799 tablet_bootstrap.cc:492] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:34.995633 28799 ts_tablet_manager.cc:1403] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.050s user 0.004s sys 0.004s
I20260618 13:59:34.997207 29025 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_706eec6806a94d3f8d5b60708a0c1cda [id=b817db93d50144d7b7c238550eb53b10] alter complete (version 1)
I20260618 13:59:34.999610 28799 raft_consensus.cc:359] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:34.999779 28799 raft_consensus.cc:385] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:34.999823 28799 raft_consensus.cc:740] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.000415 28799 consensus_queue.cc:260] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.007216 28799 ts_tablet_manager.cc:1434] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260618 13:59:35.008868 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet dd2385e6752d49429422fc854539e9c6 with delete_type TABLET_DATA_DELETED (replaced by tablet c38a2156a55e4e5eb91d101e9a61eecd) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:35.009135 28799 tablet_bootstrap.cc:492] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:35.011878 28799 tablet_bootstrap.cc:654] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:35.015655 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet aa8cf927bed9437086efd8e0f21ba26a (table default.Table_cf80fd5bbdde4cfb855c8fbe860a15f3 [id=9d63ad7327a44124af817b465a9ae0da]) successfully deleted
I20260618 13:59:35.015825 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet f3ba9186b3ff47d5a2b158fef619686e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:35.019508 29914 raft_consensus.cc:493] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.019575 28797 raft_consensus.cc:385] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.019656 29914 raft_consensus.cc:515] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.019783 28797 raft_consensus.cc:740] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.020152 28797 consensus_queue.cc:260] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.020391 29914 leader_election.cc:290] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.021092 28797 ts_tablet_manager.cc:1434] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.254s user 0.002s sys 0.000s
I20260618 13:59:35.021413 30022 raft_consensus.cc:515] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.021979 30022 leader_election.cc:290] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:35.035368 29020 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.2:52853 (request call id 117) took 1863 ms (client timeout 1000 ms). Trace:
W20260618 13:59:35.035724 29020 rpcz_store.cc:269] 0618 13:59:33.172005 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:33.177355 (+ 5350us) service_pool.cc:224] Handling call
0618 13:59:33.215973 (+ 38618us) spinlock_profiling.cc:243] Waited 35.2 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb467ef08 00007fcbab0ddca6 00007fcbab0dd9d5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:33.431024 (+215051us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:33.431030 (+ 6us) write_op.cc:454] Released schema lock
0618 13:59:35.035248 (+1604218us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:33.418558 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:33.418704 (+ 146us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:33.418710 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:33.418713 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:33.418858 (+ 145us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:33.418873 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:33.418878 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:33.418947 (+ 69us) tablet.cc:701] Row locks acquired
0618 13:59:33.418949 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:33.418990 (+ 41us) write_op.cc:270] Start()
0618 13:59:33.419026 (+ 36us) write_op.cc:276] Timestamp: P: 1781791173418945 usec, L: 0
0618 13:59:33.419030 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:33.419242 (+ 212us) log.cc:844] Serialized 530 byte log entry
0618 13:59:33.428354 (+ 9112us) op_driver.cc:464] REPLICATION: finished
0618 13:59:33.428503 (+ 149us) write_op.cc:301] APPLY: starting
0618 13:59:33.428518 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:33.428559 (+ 41us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:33.428585 (+ 26us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:33.428637 (+ 52us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:33.428669 (+ 32us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:33.428676 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:33.428753 (+ 77us) log.cc:844] Serialized 23 byte log entry
0618 13:59:33.428814 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:33.428833 (+ 19us) write_op.cc:454] Released schema lock
0618 13:59:33.428839 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":305233,"reported_tablets":198,"spinlock_wait_cycles":80923520,"child_traces":[["op",{"apply.queue_time_us":103,"apply.run_cpu_time_us":384,"apply.run_wall_time_us":384,"num_ops":1,"prepare.queue_time_us":645,"prepare.run_cpu_time_us":805,"prepare.run_wall_time_us":806,"raft.queue_time_us":942,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":271,"replication_time_us":9293}]]}
I20260618 13:59:34.991760 28484 leader_election.cc:304] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:35.039644 28484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.3:35503, user_credentials={real_user=slave}} blocked reactor thread for 47908.2us
I20260618 13:59:35.040201 29020 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_0a233b0cc52a4af38b49f5b81dbf08d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:35.041024 29020 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a233b0cc52a4af38b49f5b81dbf08d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:34.991863 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9024952ab8094fcda1329562bbde8ef5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:35.042351 28543 raft_consensus.cc:3060] T 6eea4d59935d40aa97dc10ec7469aa56 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.045499 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572d17724d249739be1a1cb9468f909" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:35.050150 28545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b95bdb0e36584d92b9c28de257553827" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:35.049341 29025 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.1:48495 (request call id 144) took 1846 ms (client timeout 1000 ms). Trace:
I20260618 13:59:35.051847 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 283523e5b22843ba935eabe6283d3a99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:35.052157 29759 raft_consensus.cc:2749] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:34.992585 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet d667ea5f029f46d89fcb3be03069b97b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.052987 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 04dac931e62a448d99633878eb70b92f with delete_type TABLET_DATA_DELETED (replaced by tablet 15483938a06e459789f87c6bb456bd15) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.053161 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 015494374645469785f1f978eaf0e9a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 2e493f647a344cd581515a77230d765f) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.053297 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet ac58e939730c4af681c2f08bcfbb6602 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.053417 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet c557afc3abfa4ef188bf248e89f49d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.053531 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 7572d17724d249739be1a1cb9468f909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:27 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.053644 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet ab521028ac1a4d0da013112d76bb169f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.053927 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 9c17e76ae9b44207998a43e3576908c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 2128dbec33db453ba51070767a15b21e) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.054085 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 3dd0eaa64271489d92aa380df877f257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:57282
W20260618 13:59:35.052841 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 35d8dd2c388e43a0be380aa953ec2c82 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 35d8dd2c388e43a0be380aa953ec2c82 already in progress: creating tablet
I20260618 13:59:35.054293 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e25027f423144b088dad955c7282921 with delete_type TABLET_DATA_DELETED (replaced by tablet a8da96fc621c47d4a444b5a39259f85d) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.054519 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 98b42d23832545ae81a7cdff19e4689f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:33 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.054680 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a0691de253b4b008e5d995fd6b876c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.054848 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 55ae29c3b7ed44cfa659130745037aa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:31 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.054991 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet 46bfe8cc2078414d84e65e22ed4ef8a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.055122 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet d3dbf13544ce408ab97834aec5862390 with delete_type TABLET_DATA_DELETED (replaced by tablet ba18e879631947f88bd266681746ec82) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.055246 28515 tablet_service.cc:1558] Processing DeleteTablet for tablet da4d4f03911f4ac099ecc2fa1d8af494 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.993314 28514 tablet_service.cc:1558] Processing DeleteTablet for tablet 97a9f949999446f09359629fe483bf67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:29 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.993788 28522 tablet_service.cc:1558] Processing DeleteTablet for tablet e5f7eaa7d218440ca2c35340c5c87814 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.994702 28524 tablet_service.cc:1558] Processing DeleteTablet for tablet 283523e5b22843ba935eabe6283d3a99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.996644 28510 tablet_service.cc:1558] Processing DeleteTablet for tablet ca2ee746c5024cfc9b31c470f9616e7d with delete_type TABLET_DATA_DELETED (replaced by tablet 60b6216474b74a14af6abf977e312b77) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.998929 28516 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f3d2f236fd44ab8af0524ea282cfa94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:34.999280 28520 tablet_service.cc:1558] Processing DeleteTablet for tablet e6de1293572f4e75af3417f281b9a142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.000025 28513 tablet_service.cc:1558] Processing DeleteTablet for tablet 591465e2b57247a2b224cc8b09225811 with delete_type TABLET_DATA_DELETED (replaced by tablet d4667fe48f0f4eff9c6c8baabdab58d4) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.000146 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 864f2602840c40fc8606b3784e671cba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.000226 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 with delete_type TABLET_DATA_DELETED (replaced by tablet cea4bc13eabc4dd98b4f96723498014c) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.000311 28512 tablet_service.cc:1558] Processing DeleteTablet for tablet 827cc0987bd541759e334fb620f04ed5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.000392 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet 7d4144b1b5ea4e968672616bc22bead8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.000487 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet 68b8293a7c924a449307ab88a1cf9290 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.000631 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet 0463fa2e30554abcbfc8f37b702e81d4 with delete_type TABLET_DATA_DELETED (replaced by tablet fe1139c409e84070a2f496a85545e8d5) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.000742 28511 tablet_service.cc:1558] Processing DeleteTablet for tablet 125bb6885ab348d885d5a3b4bdd45bc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.002215 28525 tablet_service.cc:1558] Processing DeleteTablet for tablet be59f988f0e34b9480f859cf88b9c223 with delete_type TABLET_DATA_DELETED (replaced by tablet b968a8d4014341ca9c5964263721ac79) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.002358 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet 6eea4d59935d40aa97dc10ec7469aa56 with delete_type TABLET_DATA_DELETED (replaced by tablet 4bc87770baa34842a25ed97a19354446) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.002487 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet 02b5cc08ca0c4e719e0414b345211ba5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.002575 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet c7f42997416841a7b2c1ad347d2e9282 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.002628 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet c552d1316e3340f4bcfa39618ae309b8 with delete_type TABLET_DATA_DELETED (replaced by tablet fe750fc5d998448caeab0a6c808dfba7) from {username='slave'} at 127.0.0.1:57282
W20260618 13:59:35.057190 28346 leader_election.cc:343] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.057336 28346 leader_election.cc:304] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:35.057775 30022 raft_consensus.cc:2749] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:35.007059 29746 tablet_replica.cc:333] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:35.058213 29746 raft_consensus.cc:2243] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:35.058406 29746 raft_consensus.cc:2272] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:35.058678 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet e3d8616846c649b6a969b971db0d03db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.059023 29746 ts_tablet_manager.cc:1916] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:35.060756 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet cd06bb7cffc34412b420d70435fce373 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.062920 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet f278d0acbb0c4a0ca61640bc142e6335 with delete_type TABLET_DATA_DELETED (replaced by tablet 72c04f55890648f0a8184f3783243f7c) from {username='slave'} at 127.0.0.1:57282
W20260618 13:59:35.062016 29018 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.3:42065 (request call id 142) took 2884 ms (client timeout 1000 ms). Trace:
I20260618 13:59:35.070842 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be00bfa9bc1a4b83b18bfad05ba1495e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:35.038022 28539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba53ad1988064049a61fcca3a85f8779" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:35.071578 28346 leader_election.cc:343] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.038936 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0463fa2e30554abcbfc8f37b702e81d4" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:35.076099 28346 leader_election.cc:343] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.022259 29719 raft_consensus.cc:515] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.078269 29719 leader_election.cc:290] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.046605 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba53ad1988064049a61fcca3a85f8779" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:35.047183 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0463fa2e30554abcbfc8f37b702e81d4" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:35.078593 28680 raft_consensus.cc:3060] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.057493 30008 raft_consensus.cc:493] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.078894 30008 raft_consensus.cc:515] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.079120 28797 tablet_bootstrap.cc:492] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:35.079398 30008 leader_election.cc:290] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:35.067690 28618 leader_election.cc:343] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:35.079807 28618 consensus_peers.cc:597] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:35.080011 28618 leader_election.cc:343] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:35.080487 28618 consensus_peers.cc:597] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aa8cf927bed9437086efd8e0f21ba26a. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:35.080555 28797 tablet_bootstrap.cc:654] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:35.080721 28618 leader_election.cc:343] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.082784 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09cbe2337dd14da1b8e0629ded7e7ad0" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:35.083554 28346 leader_election.cc:343] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.083683 28346 leader_election.cc:304] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:35.084105 30022 raft_consensus.cc:2749] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:35.084645 28539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "015494374645469785f1f978eaf0e9a5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:35.085824 30019 tablet.cc:1724] T 15180aefba0544329daee74308ef5010 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:35.088060 30027 raft_consensus.cc:493] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.088299 30027 raft_consensus.cc:515] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.093051 28798 tablet_bootstrap.cc:492] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:35.093395 28798 ts_tablet_manager.cc:1403] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.345s user 0.004s sys 0.000s
W20260618 13:59:35.064972 29018 rpcz_store.cc:269] 0618 13:59:32.176066 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.256417 (+ 80351us) service_pool.cc:224] Handling call
0618 13:59:35.060933 (+2804516us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:32.987905 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:32.987979 (+ 74us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:32.987984 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:32.987988 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:32.988121 (+ 133us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:32.988131 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:32.988137 (+ 6us) tablet.cc:685] Acquiring locks for 3 operations
0618 13:59:32.988209 (+ 72us) tablet.cc:701] Row locks acquired
0618 13:59:32.988211 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:32.988235 (+ 24us) write_op.cc:270] Start()
0618 13:59:32.988258 (+ 23us) write_op.cc:276] Timestamp: P: 1781791172987848 usec, L: 0
0618 13:59:32.988261 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:32.988395 (+ 134us) log.cc:844] Serialized 1284 byte log entry
0618 13:59:33.158888 (+170493us) op_driver.cc:464] REPLICATION: finished
0618 13:59:33.169598 (+ 10710us) write_op.cc:301] APPLY: starting
0618 13:59:33.169612 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:33.169658 (+ 46us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:33.169693 (+ 35us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:33.169763 (+ 70us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:33.169792 (+ 29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0618 13:59:33.169798 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:33.169860 (+ 62us) log.cc:844] Serialized 35 byte log entry
0618 13:59:33.169914 (+ 54us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:33.169930 (+ 16us) write_op.cc:454] Released schema lock
0618 13:59:33.169938 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:33.170005 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:33.170008 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":15703,"reported_tablets":192,"child_traces":[["op",{"apply.queue_time_us":10665,"apply.run_cpu_time_us":398,"apply.run_wall_time_us":398,"mutex_wait_us":4424,"num_ops":3,"prepare.queue_time_us":4805,"prepare.run_cpu_time_us":538,"prepare.run_wall_time_us":537,"replication_time_us":170588}]]}
I20260618 13:59:35.116379 28411 raft_consensus.cc:2468] T b95bdb0e36584d92b9c28de257553827 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:35.116448 29745 ts_tablet_manager.cc:1929] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:35.116532 29745 log.cc:1199] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/c7f42997416841a7b2c1ad347d2e9282
I20260618 13:59:35.116941 29745 ts_tablet_manager.cc:1950] T c7f42997416841a7b2c1ad347d2e9282 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:35.119127 28409 raft_consensus.cc:2468] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:35.119926 30029 raft_consensus.cc:515] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.120576 30029 leader_election.cc:290] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.121327 29640 raft_consensus.cc:515] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.121876 29640 leader_election.cc:290] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:35.122251 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 57) took 3079 ms. Trace:
I20260618 13:59:35.122320 29566 rpcz_store.cc:276] 0618 13:59:32.043245 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.043293 (+ 48us) service_pool.cc:224] Handling call
0618 13:59:34.748580 (+2705287us) ts_tablet_manager.cc:1151] Deleting tablet d6871232860b41d489e34d95eb897241
0618 13:59:34.748591 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:34.837935 (+ 89344us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:34.927714 (+ 89779us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:35.122204 (+194490us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2705143}
I20260618 13:59:35.122671 29566 tablet_replica.cc:333] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:35.123489 28682 raft_consensus.cc:2468] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:35.124840 28680 raft_consensus.cc:2468] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:35.126500 28346 leader_election.cc:304] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.126894 30022 raft_consensus.cc:2804] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:35.127025 30022 raft_consensus.cc:697] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:35.127250 30022 consensus_queue.cc:237] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.128101 28346 leader_election.cc:304] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.128415 29914 raft_consensus.cc:2804] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:35.128504 29914 raft_consensus.cc:697] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:35.128688 29914 consensus_queue.cc:237] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.130493 29749 raft_consensus.cc:493] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.137059 29749 raft_consensus.cc:515] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.137822 29749 leader_election.cc:290] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:35.139070 28483 leader_election.cc:343] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 125bb6885ab348d885d5a3b4bdd45bc7
I20260618 13:59:35.132879 29746 ts_tablet_manager.cc:1929] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:35.139609 29746 log.cc:1199] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/0fae928833e44d2ba860d6184d4c0142
I20260618 13:59:35.139670 29679 raft_consensus.cc:493] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.139838 29679 raft_consensus.cc:515] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.140033 29746 ts_tablet_manager.cc:1950] T 0fae928833e44d2ba860d6184d4c0142 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:35.140574 29746 tablet_replica.cc:333] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:35.140605 29679 leader_election.cc:290] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:35.140780 29746 raft_consensus.cc:2243] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:35.140971 29746 raft_consensus.cc:2272] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:35.141678 29746 ts_tablet_manager.cc:1916] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:35.145727 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe1139c409e84070a2f496a85545e8d5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:35.146572 28346 leader_election.cc:343] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.147881 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe1139c409e84070a2f496a85545e8d5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:35.148347 28346 leader_election.cc:343] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.148442 28346 leader_election.cc:304] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:35.148898 29679 raft_consensus.cc:2749] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:35.167600 28798 raft_consensus.cc:359] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.167882 28798 raft_consensus.cc:385] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.168207 28798 raft_consensus.cc:740] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.168871 28798 consensus_queue.cc:260] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:35.174718 28618 leader_election.cc:343] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.175061 30027 leader_election.cc:290] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:35.175107 28618 leader_election.cc:343] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.177832 29679 raft_consensus.cc:493] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.178018 29679 raft_consensus.cc:515] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:35.178655 28617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 208833us
I20260618 13:59:35.178771 29679 leader_election.cc:290] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:35.180575 28346 leader_election.cc:343] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 3a0039a463ec4a9c82a21a3a69b565fc
W20260618 13:59:35.180940 28484 leader_election.cc:343] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 125bb6885ab348d885d5a3b4bdd45bc7
I20260618 13:59:35.181206 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0039a463ec4a9c82a21a3a69b565fc" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:35.181751 28346 leader_election.cc:343] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.182006 28346 leader_election.cc:304] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:35.182637 29679 raft_consensus.cc:2749] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:35.184930 28406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09cbe2337dd14da1b8e0629ded7e7ad0" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:35.185225 28406 raft_consensus.cc:2468] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:35.185585 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d3d363766546c4a97b0763bad11f71" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:35.185803 28402 raft_consensus.cc:3060] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.186800 29026 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 88) took 1973 ms. Trace:
I20260618 13:59:35.187381 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "015494374645469785f1f978eaf0e9a5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:35.187630 28407 raft_consensus.cc:3060] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.187136 29026 rpcz_store.cc:276] 0618 13:59:33.213011 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.662116 (+1449105us) service_pool.cc:224] Handling call
0618 13:59:34.727211 (+ 65095us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:34.771569 (+ 44358us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:35.077371 (+305802us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:35.186576 (+109205us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:35.186682 (+ 106us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:34.820570 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:34.820743 (+ 173us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:34.820751 (+ 8us) write_op.cc:435] Acquired schema lock
0618 13:59:34.820754 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:34.820885 (+ 131us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:34.820899 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:34.820905 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:34.821030 (+ 125us) tablet.cc:701] Row locks acquired
0618 13:59:34.821034 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:34.821083 (+ 49us) write_op.cc:270] Start()
0618 13:59:34.821131 (+ 48us) write_op.cc:276] Timestamp: P: 1781791174821052 usec, L: 0
0618 13:59:34.821136 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:34.824890 (+ 3754us) log.cc:844] Serialized 433 byte log entry
0618 13:59:35.076319 (+251429us) op_driver.cc:464] REPLICATION: finished
0618 13:59:35.076443 (+ 124us) write_op.cc:301] APPLY: starting
0618 13:59:35.076461 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:35.076519 (+ 58us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:35.076550 (+ 31us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:35.076672 (+ 122us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:35.076733 (+ 61us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:35.076744 (+ 11us) write_op.cc:312] APPLY: finished
0618 13:59:35.076853 (+ 109us) log.cc:844] Serialized 29 byte log entry
0618 13:59:35.076923 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.076946 (+ 23us) write_op.cc:454] Released schema lock
0618 13:59:35.076952 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:35.077023 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.077027 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":70,"apply.run_cpu_time_us":572,"apply.run_wall_time_us":571,"num_ops":2,"prepare.queue_time_us":4511,"prepare.run_cpu_time_us":994,"prepare.run_wall_time_us":4446,"replication_time_us":255145}]]}
I20260618 13:59:35.188982 28402 raft_consensus.cc:2468] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:35.189177 29023 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 90) took 1923 ms. Trace:
I20260618 13:59:35.189391 29023 rpcz_store.cc:276] 0618 13:59:33.265748 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.962281 (+1696533us) service_pool.cc:224] Handling call
0618 13:59:35.189121 (+226840us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260618 13:59:35.190601 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ed88c142d7d40cda6392c06bc0486ca" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:35.190925 28404 raft_consensus.cc:2468] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:35.191205 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d523419ff244f085d6eac1a3231973" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:35.191521 28408 raft_consensus.cc:3060] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.191869 28617 leader_election.cc:304] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.192370 30059 raft_consensus.cc:2804] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:35.192430 30059 raft_consensus.cc:493] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:35.192525 30059 raft_consensus.cc:3060] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.194585 28408 raft_consensus.cc:2468] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:35.195030 29679 raft_consensus.cc:493] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.194978 30059 raft_consensus.cc:515] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.195286 29679 raft_consensus.cc:515] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.195672 30059 leader_election.cc:290] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.195869 29679 leader_election.cc:290] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.195999 28617 leader_election.cc:304] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.196532 30059 raft_consensus.cc:2804] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:35.196710 30059 raft_consensus.cc:697] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:35.196924 30059 consensus_queue.cc:237] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.198714 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d1073b726ef4cbeac6c874fc8863e90" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:35.199275 28346 leader_election.cc:343] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.201287 28407 raft_consensus.cc:2468] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:35.202157 29679 raft_consensus.cc:493] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.202289 29679 raft_consensus.cc:515] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.202919 29679 leader_election.cc:290] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:35.202944 29566 raft_consensus.cc:2243] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:35.203097 29566 raft_consensus.cc:2272] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:35.203436 28617 leader_election.cc:304] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.203666 29566 ts_tablet_manager.cc:1916] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:35.203871 29303 raft_consensus.cc:2804] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:35.203914 28617 leader_election.cc:304] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.203992 29303 raft_consensus.cc:697] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:35.204233 29303 consensus_queue.cc:237] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.207062 30060 raft_consensus.cc:2804] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:35.207139 30060 raft_consensus.cc:493] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:35.207208 30060 raft_consensus.cc:3060] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.207196 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a9f949999446f09359629fe483bf67" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:35.207715 28346 leader_election.cc:343] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.211123 30060 raft_consensus.cc:515] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.211645 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a9f949999446f09359629fe483bf67" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:35.211880 30060 leader_election.cc:290] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:35.212095 28346 leader_election.cc:343] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.212230 28346 leader_election.cc:304] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:35.212662 29679 raft_consensus.cc:2749] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:35.213353 28617 consensus_peers.cc:597] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 283523e5b22843ba935eabe6283d3a99. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:35.213423 29566 ts_tablet_manager.cc:1929] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:35.214252 29566 log.cc:1199] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/eead369de58f40378254697cbbbc6958
I20260618 13:59:35.214754 29566 ts_tablet_manager.cc:1950] T eead369de58f40378254697cbbbc6958 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:35.215188 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 58) took 3171 ms. Trace:
I20260618 13:59:35.215351 29566 rpcz_store.cc:276] 0618 13:59:32.043991 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.044037 (+ 46us) service_pool.cc:224] Handling call
0618 13:59:35.122628 (+3078591us) ts_tablet_manager.cc:1151] Deleting tablet eead369de58f40378254697cbbbc6958
0618 13:59:35.122637 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:35.208061 (+ 85424us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:35.213366 (+ 5305us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:35.215168 (+ 1802us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80066,"tablet-delete.queue_time_us":3078437}
I20260618 13:59:35.215857 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 59) took 3171 ms. Trace:
I20260618 13:59:35.216197 29566 rpcz_store.cc:276] 0618 13:59:32.044810 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.044868 (+ 58us) service_pool.cc:224] Handling call
0618 13:59:35.215773 (+3170905us) ts_tablet_manager.cc:1151] Deleting tablet edc72a06ee8e4420b07a17d2feb3fcc2
0618 13:59:35.215782 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:35.215843 (+ 61us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3170732}
I20260618 13:59:35.216629 29566 tablet_replica.cc:333] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:35.216931 29566 raft_consensus.cc:2243] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:35.217198 29566 pending_rounds.cc:70] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257: Trying to abort 1 pending ops.
I20260618 13:59:35.217316 29566 pending_rounds.cc:77] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298216648842285056 op_type: NO_OP noop_request { }
I20260618 13:59:35.217481 29566 raft_consensus.cc:2889] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:35.217633 29566 raft_consensus.cc:2272] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:35.214735 28617 leader_election.cc:304] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:35.218930 30060 raft_consensus.cc:2804] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:35.219038 30060 raft_consensus.cc:697] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:35.219278 30060 consensus_queue.cc:237] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.223088 29566 ts_tablet_manager.cc:1916] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:35.223909 28617 leader_election.cc:304] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:35.224150 28617 leader_election.cc:304] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:35.224160 29303 raft_consensus.cc:2804] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:35.224413 29303 raft_consensus.cc:493] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:35.224488 29303 raft_consensus.cc:3060] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.224471 30027 raft_consensus.cc:2804] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:35.224684 30027 raft_consensus.cc:697] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:35.224936 30027 consensus_queue.cc:237] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:35.052361 29025 rpcz_store.cc:269] 0618 13:59:33.199681 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:33.232611 (+ 32930us) service_pool.cc:224] Handling call
0618 13:59:33.255131 (+ 22520us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb467ef08 00007fcbab0ddca6 00007fcbab0dd9d5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:33.674687 (+419556us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:33.674693 (+ 6us) write_op.cc:454] Released schema lock
0618 13:59:33.693075 (+ 18382us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:33.693081 (+ 6us) write_op.cc:454] Released schema lock
0618 13:59:34.164129 (+471048us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.164135 (+ 6us) write_op.cc:454] Released schema lock
0618 13:59:35.045860 (+881725us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:33.668013 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:33.668125 (+ 112us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:33.668132 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:33.668135 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:33.668242 (+ 107us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:33.668254 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:33.668259 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:33.668387 (+ 128us) tablet.cc:701] Row locks acquired
0618 13:59:33.668389 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:33.668424 (+ 35us) write_op.cc:270] Start()
0618 13:59:33.668456 (+ 32us) write_op.cc:276] Timestamp: P: 1781791173668356 usec, L: 0
0618 13:59:33.668459 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:33.668646 (+ 187us) log.cc:844] Serialized 907 byte log entry
0618 13:59:33.672266 (+ 3620us) op_driver.cc:464] REPLICATION: finished
0618 13:59:33.672418 (+ 152us) write_op.cc:301] APPLY: starting
0618 13:59:33.672432 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:33.672474 (+ 42us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:33.672504 (+ 30us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:33.672565 (+ 61us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:33.672597 (+ 32us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:33.672604 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:33.672674 (+ 70us) log.cc:844] Serialized 29 byte log entry
0618 13:59:33.672728 (+ 54us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:33.672747 (+ 19us) write_op.cc:454] Released schema lock
0618 13:59:33.672754 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0618 13:59:33.686548 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:33.686665 (+ 117us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:33.686670 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:33.686672 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:33.686768 (+ 96us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:33.686781 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:33.686785 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:33.686844 (+ 59us) tablet.cc:701] Row locks acquired
0618 13:59:33.686846 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:33.686877 (+ 31us) write_op.cc:270] Start()
0618 13:59:33.686925 (+ 48us) write_op.cc:276] Timestamp: P: 1781791173686808 usec, L: 0
0618 13:59:33.686929 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:33.687124 (+ 195us) log.cc:844] Serialized 356 byte log entry
0618 13:59:33.690733 (+ 3609us) op_driver.cc:464] REPLICATION: finished
0618 13:59:33.690878 (+ 145us) write_op.cc:301] APPLY: starting
0618 13:59:33.690893 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:33.690933 (+ 40us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:33.690957 (+ 24us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:33.691005 (+ 48us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:33.691035 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:33.691042 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:33.691122 (+ 80us) log.cc:844] Serialized 23 byte log entry
0618 13:59:33.691181 (+ 59us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:33.691197 (+ 16us) write_op.cc:454] Released schema lock
0618 13:59:33.691203 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0618 13:59:33.983135 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:33.983247 (+ 112us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:33.983253 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:33.983255 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:33.983353 (+ 98us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:33.983366 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:33.983369 (+ 3us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:33.983454 (+ 85us) tablet.cc:701] Row locks acquired
0618 13:59:33.983458 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:33.983507 (+ 49us) write_op.cc:270] Start()
0618 13:59:33.983554 (+ 47us) write_op.cc:276] Timestamp: P: 1781791173983454 usec, L: 0
0618 13:59:33.983560 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:33.983864 (+ 304us) log.cc:844] Serialized 356 byte log entry
0618 13:59:33.991744 (+ 7880us) op_driver.cc:464] REPLICATION: finished
0618 13:59:33.991880 (+ 136us) write_op.cc:301] APPLY: starting
0618 13:59:33.991900 (+ 20us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:33.991947 (+ 47us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:33.991996 (+ 49us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:33.992055 (+ 59us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:33.992088 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:33.992095 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:33.992189 (+ 94us) log.cc:844] Serialized 23 byte log entry
0618 13:59:33.992266 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:33.992295 (+ 29us) write_op.cc:454] Released schema lock
0618 13:59:33.992306 (+ 11us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:33.992393 (+ 87us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:33.992400 (+ 7us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:34.128808 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:34.128951 (+ 143us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:34.128960 (+ 9us) write_op.cc:435] Acquired schema lock
0618 13:59:34.128965 (+ 5us) tablet.cc:662] Decoding operations
0618 13:59:34.129095 (+ 130us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:34.129111 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:34.129117 (+ 6us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:34.129193 (+ 76us) tablet.cc:701] Row locks acquired
0618 13:59:34.129197 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:34.129246 (+ 49us) write_op.cc:270] Start()
0618 13:59:34.129291 (+ 45us) write_op.cc:276] Timestamp: P: 1781791174129177 usec, L: 0
0618 13:59:34.129296 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:34.129540 (+ 244us) log.cc:844] Serialized 356 byte log entry
0618 13:59:34.138844 (+ 9304us) op_driver.cc:464] REPLICATION: finished
0618 13:59:34.138960 (+ 116us) write_op.cc:301] APPLY: starting
0618 13:59:34.138977 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:34.139020 (+ 43us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:34.139045 (+ 25us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:34.139106 (+ 61us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:34.139140 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:34.139147 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:34.139217 (+ 70us) log.cc:844] Serialized 23 byte log entry
0618 13:59:34.139274 (+ 57us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.139293 (+ 19us) write_op.cc:454] Released schema lock
0618 13:59:34.139301 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0618 13:59:34.460558 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:34.484464 (+ 23906us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:34.484470 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:34.484472 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:34.484572 (+ 100us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:34.484582 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:34.484586 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:34.498554 (+ 13968us) tablet.cc:701] Row locks acquired
0618 13:59:34.498558 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:34.498624 (+ 66us) write_op.cc:270] Start()
0618 13:59:34.503246 (+ 4622us) write_op.cc:276] Timestamp: P: 1781791174498576 usec, L: 0
0618 13:59:34.503251 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:34.503461 (+ 210us) log.cc:844] Serialized 356 byte log entry
0618 13:59:34.712703 (+209242us) op_driver.cc:464] REPLICATION: finished
0618 13:59:34.712833 (+ 130us) write_op.cc:301] APPLY: starting
0618 13:59:34.712857 (+ 24us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:34.712913 (+ 56us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:34.712967 (+ 54us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:34.713038 (+ 71us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:34.713084 (+ 46us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:34.713095 (+ 11us) write_op.cc:312] APPLY: finished
0618 13:59:34.713184 (+ 89us) log.cc:844] Serialized 23 byte log entry
0618 13:59:34.713246 (+ 62us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.713268 (+ 22us) write_op.cc:454] Released schema lock
0618 13:59:34.713276 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:34.713351 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.713355 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:34.718857 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:34.719013 (+ 156us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:34.719022 (+ 9us) write_op.cc:435] Acquired schema lock
0618 13:59:34.719026 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:34.719156 (+ 130us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:34.719173 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:34.719180 (+ 7us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:34.719260 (+ 80us) tablet.cc:701] Row locks acquired
0618 13:59:34.719264 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:34.719302 (+ 38us) write_op.cc:270] Start()
0618 13:59:34.719341 (+ 39us) write_op.cc:276] Timestamp: P: 1781791174719234 usec, L: 0
0618 13:59:34.719347 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:34.719558 (+ 211us) log.cc:844] Serialized 356 byte log entry
0618 13:59:34.996170 (+276612us) op_driver.cc:464] REPLICATION: finished
0618 13:59:34.996300 (+ 130us) write_op.cc:301] APPLY: starting
0618 13:59:34.996321 (+ 21us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:34.996379 (+ 58us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:34.996412 (+ 33us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:34.996473 (+ 61us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:34.996508 (+ 35us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:34.996514 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:34.996605 (+ 91us) log.cc:844] Serialized 23 byte log entry
0618 13:59:34.996677 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.996699 (+ 22us) write_op.cc:454] Released schema lock
0618 13:59:34.996708 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:34.996811 (+ 103us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.996816 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":32806,"reported_tablets":178,"spinlock_wait_cycles":39123200,"child_traces":[["op",{"apply.queue_time_us":107,"apply.run_cpu_time_us":386,"apply.run_wall_time_us":384,"num_ops":2,"prepare.queue_time_us":43,"prepare.run_cpu_time_us":738,"prepare.run_wall_time_us":738,"raft.queue_time_us":855,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":276,"replication_time_us":3779}],["op",{"apply.queue_time_us":103,"apply.run_cpu_time_us":370,"apply.run_wall_time_us":369,"num_ops":1,"prepare.queue_time_us":8373,"prepare.run_cpu_time_us":687,"prepare.run_wall_time_us":688,"raft.queue_time_us":855,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":273,"replication_time_us":3778}],["op",{"apply.queue_time_us":68,"apply.run_cpu_time_us":498,"apply.run_wall_time_us":497,"num_ops":1,"prepare.queue_time_us":64,"prepare.run_cpu_time_us":893,"prepare.run_wall_time_us":893,"raft.queue_time_us":101,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":299,"replication_time_us":8149}],["op",{"apply.queue_time_us":69,"apply.run_cpu_time_us":391,"apply.run_wall_time_us":390,"num_ops":1,"prepare.queue_time_us":2765,"prepare.run_cpu_time_us":846,"prepare.run_wall_time_us":847,"raft.queue_time_us":1480,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":285,"replication_time_us":9515}],["op",{"apply.queue_time_us":69,"apply.run_cpu_time_us":512,"apply.run_wall_time_us":512,"num_ops":1,"prepare.queue_time_us":6668,"prepare.run_cpu_time_us":891,"prepare.run_wall_time_us":42977,"replication_time_us":209415}],["op",{"apply.queue_time_us":73,"apply.run_cpu_time_us":498,"apply.run_wall_time_us":498,"num_ops":1,"prepare.queue_time_us":70,"prepare.run_cpu_time_us":769,"prepare.run_wall_time_us":770,"replication_time_us":276780}]]}
I20260618 13:59:35.227669 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 56) took 2321 ms. Trace:
I20260618 13:59:35.227814 29745 rpcz_store.cc:276] 0618 13:59:32.905873 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.907469 (+ 1596us) service_pool.cc:224] Handling call
0618 13:59:34.771192 (+1863723us) ts_tablet_manager.cc:1151] Deleting tablet c7f42997416841a7b2c1ad347d2e9282
0618 13:59:34.771202 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:34.854817 (+ 83615us) spinlock_profiling.cc:243] Waited 82.9 ms on lock 0x6160000e5c54. stack: 00007f31ce917496 00007f31ce917250 00007f31cd9423ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007f31da9bd7e0 00007f31dbd09933 00007f31de8ccae7 00007f31de8ffbb4 00007f31de8e05d0 00007f31de8e035d 0000000000aa681c 00007f31ce95d9ff 00007f31ce960d1b 00007f31ce960b98
0618 13:59:35.024712 (+169895us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:35.116396 (+ 91684us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:35.227650 (+111254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190762880,"tablet-delete.queue_time_us":1863558}
I20260618 13:59:35.228235 29745 tablet_replica.cc:333] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:35.228477 29745 raft_consensus.cc:2243] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:35.228502 29303 raft_consensus.cc:515] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.228669 29745 raft_consensus.cc:2272] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:35.229182 29303 leader_election.cc:290] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.229257 29745 ts_tablet_manager.cc:1916] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:35.224417 28617 consensus_peers.cc:597] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dd2385e6752d49429422fc854539e9c6. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:35.230858 28617 leader_election.cc:304] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.231032 29303 raft_consensus.cc:2804] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:35.231124 29303 raft_consensus.cc:697] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:35.231349 29303 consensus_queue.cc:237] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.231518 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet c7f42997416841a7b2c1ad347d2e9282 (table default.Table_0a13b90182654346acbbc8f47a56436d [id=c31d65adb55f458c9ced14d598464f89]) successfully deleted
I20260618 13:59:35.233011 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572d17724d249739be1a1cb9468f909" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:35.233215 28407 raft_consensus.cc:3060] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.235996 28407 raft_consensus.cc:2468] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:35.237020 29745 ts_tablet_manager.cc:1929] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:35.237237 29745 log.cc:1199] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/d502d67826ff4d7a8b0f515e5d8b6d77
I20260618 13:59:35.237577 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09cbe2337dd14da1b8e0629ded7e7ad0" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:35.237761 29745 ts_tablet_manager.cc:1950] T d502d67826ff4d7a8b0f515e5d8b6d77 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:35.237829 28407 raft_consensus.cc:3060] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.238124 29679 raft_consensus.cc:493] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.238270 29679 raft_consensus.cc:515] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.239454 29679 leader_election.cc:290] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.240638 28407 raft_consensus.cc:2468] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:35.240725 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054a7afb3b644aa49df78302cf8f02c9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:35.241315 28346 leader_election.cc:343] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.241434 28484 leader_election.cc:304] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:35.241829 28484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.3:35503, user_credentials={real_user=slave}} blocked reactor thread for 60878us
I20260618 13:59:35.242303 28799 tablet_bootstrap.cc:492] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:35.242496 28799 ts_tablet_manager.cc:1403] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.233s user 0.014s sys 0.030s
I20260618 13:59:35.243186 28799 raft_consensus.cc:359] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.243346 28799 raft_consensus.cc:385] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.243319 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ed88c142d7d40cda6392c06bc0486ca" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:35.243422 28799 raft_consensus.cc:740] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.243637 28799 consensus_queue.cc:260] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.244596 28540 raft_consensus.cc:3060] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.244781 28799 ts_tablet_manager.cc:1434] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:35.244776 29755 raft_consensus.cc:2749] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:35.245106 28799 tablet_bootstrap.cc:492] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:35.249037 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d3d363766546c4a97b0763bad11f71" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:35.249439 28539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d523419ff244f085d6eac1a3231973" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:35.250883 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09cbe2337dd14da1b8e0629ded7e7ad0" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:35.251123 29759 raft_consensus.cc:493] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.251282 29759 raft_consensus.cc:515] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.251902 29759 leader_election.cc:290] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:35.252163 28546 raft_consensus.cc:3060] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.252861 28617 leader_election.cc:304] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:35.253065 29303 raft_consensus.cc:2804] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:35.253161 29303 raft_consensus.cc:697] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
W20260618 13:59:35.253185 28483 leader_election.cc:343] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 68b8293a7c924a449307ab88a1cf9290
I20260618 13:59:35.253376 29303 consensus_queue.cc:237] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.253787 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d1073b726ef4cbeac6c874fc8863e90" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:35.252120 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054a7afb3b644aa49df78302cf8f02c9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:35.254554 28346 leader_election.cc:343] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.254803 28346 leader_election.cc:304] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:35.255430 28346 leader_election.cc:343] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.255609 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ed88c142d7d40cda6392c06bc0486ca" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:35.255654 28346 leader_election.cc:304] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:35.255813 28409 raft_consensus.cc:3060] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.256034 29984 raft_consensus.cc:2749] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:35.256330 29984 raft_consensus.cc:2749] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:35.257108 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ed88c142d7d40cda6392c06bc0486ca" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:35.259305 28618 leader_election.cc:343] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:35.259727 28618 leader_election.cc:343] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:35.260007 28618 leader_election.cc:343] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.261741 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 57) took 2355 ms. Trace:
I20260618 13:59:35.261968 29745 rpcz_store.cc:276] 0618 13:59:32.905961 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.907661 (+ 1700us) service_pool.cc:224] Handling call
0618 13:59:35.228166 (+2320505us) ts_tablet_manager.cc:1151] Deleting tablet d502d67826ff4d7a8b0f515e5d8b6d77
0618 13:59:35.228176 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:35.234043 (+ 5867us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:35.236951 (+ 2908us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:35.261632 (+ 24681us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2320381}
I20260618 13:59:35.262738 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet d502d67826ff4d7a8b0f515e5d8b6d77 (table default.Table_b5800aca80594ec1bd3a86ec47dbfc64 [id=2710734517a24ad6baf0d47bf23e9bdf]) successfully deleted
I20260618 13:59:35.262887 29745 tablet_replica.cc:333] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:35.263229 29745 raft_consensus.cc:2243] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:35.263813 29745 raft_consensus.cc:2272] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:35.264217 28617 leader_election.cc:304] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:35.264570 29745 ts_tablet_manager.cc:1916] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:35.264423 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572d17724d249739be1a1cb9468f909" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:35.265143 30059 raft_consensus.cc:2804] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:35.265254 30059 raft_consensus.cc:697] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:35.265481 30059 consensus_queue.cc:237] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:35.271199 28618 leader_election.cc:343] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:35.272701 28618 leader_election.cc:343] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:35.274310 28618 leader_election.cc:343] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:35.274941 28618 consensus_peers.cc:597] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:35.274909 29018 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_995e1233361c4c2aae5e3ad5c9c989a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:35.275436 28484 leader_election.cc:343] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 68b8293a7c924a449307ab88a1cf9290
W20260618 13:59:35.275682 29018 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_995e1233361c4c2aae5e3ad5c9c989a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:35.275722 28484 leader_election.cc:304] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:35.276144 29749 raft_consensus.cc:2749] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:35.277537 29026 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_e9f8c36af8a047f09b3fa4cf54456229"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:35.278366 29026 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e9f8c36af8a047f09b3fa4cf54456229 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:35.278807 28409 raft_consensus.cc:2468] T 8ed88c142d7d40cda6392c06bc0486ca P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:35.280632 29023 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_bb918931281942b486860c5e4f5755f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:35.281585 29023 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bb918931281942b486860c5e4f5755f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:35.279917 28617 leader_election.cc:304] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.282987 29714 raft_consensus.cc:2804] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:35.283129 29714 raft_consensus.cc:697] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:35.283438 29714 consensus_queue.cc:237] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.284742 28797 tablet_bootstrap.cc:492] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:35.284876 28797 ts_tablet_manager.cc:1403] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.206s user 0.003s sys 0.000s
I20260618 13:59:35.285784 28797 raft_consensus.cc:359] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.285933 28797 raft_consensus.cc:385] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.285982 28797 raft_consensus.cc:740] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.286181 28797 consensus_queue.cc:260] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:35.287091 28989 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet c25fa765ec794c34b0aa43761961a8c9 consensus state has no leader
I20260618 13:59:35.287576 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 0fae928833e44d2ba860d6184d4c0142 (table default.Table_9e1e848ca5c94952ab3928c5feafe2f1 [id=14652783e7f54ebcb5d7ba01d1855118]) successfully deleted
I20260618 13:59:35.287876 28797 ts_tablet_manager.cc:1434] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:35.288120 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet d6871232860b41d489e34d95eb897241 (table default.Table_c18d75e5991b440caf136e9248a572da [id=5a3cb5befb614a70b4ecf8ccfedf8ecd]) successfully deleted
I20260618 13:59:35.288223 28797 tablet_bootstrap.cc:492] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:35.288544 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet eead369de58f40378254697cbbbc6958 (table default.Table_e8ae99dfd8e84d7bb2723eb5fe129c2f [id=6d8f3597814241f3bc8c920d3a59008e]) successfully deleted
W20260618 13:59:35.289062 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet edc72a06ee8e4420b07a17d2feb3fcc2 because the tablet was not found. No further retry: Not found: Tablet not found: edc72a06ee8e4420b07a17d2feb3fcc2
I20260618 13:59:35.290041 28797 tablet_bootstrap.cc:654] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:35.294504 28798 ts_tablet_manager.cc:1434] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260618 13:59:35.296700 28799 tablet_bootstrap.cc:654] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:35.306217 28797 tablet_bootstrap.cc:492] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:35.306380 28797 ts_tablet_manager.cc:1403] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.001s
I20260618 13:59:35.307098 28797 raft_consensus.cc:359] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.307272 28797 raft_consensus.cc:385] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.307124 29025 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_9b57c986604640a3b9a634d8f979787e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:35.307320 28797 raft_consensus.cc:740] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.307540 28797 consensus_queue.cc:260] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:35.307830 29025 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9b57c986604640a3b9a634d8f979787e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:35.308097 28797 ts_tablet_manager.cc:1434] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 13:59:35.308337 28797 tablet_bootstrap.cc:492] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:35.309407 28484 consensus_peers.cc:597] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 -> Peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Couldn't send request to peer d6a169b16ced46ca9cfbc631709bb257. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8fd9fa9675234db2abff694a517c0ea8. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:35.310204 28797 tablet_bootstrap.cc:654] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:35.311575 28799 tablet_bootstrap.cc:492] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:35.311734 28799 ts_tablet_manager.cc:1403] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.067s user 0.000s sys 0.006s
I20260618 13:59:35.312495 28799 raft_consensus.cc:359] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.312682 28799 raft_consensus.cc:385] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.312754 28799 raft_consensus.cc:740] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.313038 28799 consensus_queue.cc:260] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.313531 28799 ts_tablet_manager.cc:1434] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 13:59:35.313832 28799 tablet_bootstrap.cc:492] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:35.315515 28799 tablet_bootstrap.cc:654] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:35.323622 28799 tablet_bootstrap.cc:492] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:35.323793 28799 ts_tablet_manager.cc:1403] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.003s
I20260618 13:59:35.325078 28799 raft_consensus.cc:359] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.325270 28799 raft_consensus.cc:385] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.325354 28799 raft_consensus.cc:740] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.327059 28386 tablet_service.cc:1511] Processing CreateTablet for tablet 60e02f51b8624a739fadbf4fafe90cb5 (DEFAULT_TABLE table=default.Table_84a1abff52a348629514e93e578bfac3 [id=db6c4f85b83849128ab1b1447e7357be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.327128 28799 consensus_queue.cc:260] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.327729 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60e02f51b8624a739fadbf4fafe90cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.329532 28797 tablet_bootstrap.cc:492] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:35.329805 28799 ts_tablet_manager.cc:1434] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260618 13:59:35.330154 28799 tablet_bootstrap.cc:492] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:35.330754 28797 ts_tablet_manager.cc:1403] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.004s
I20260618 13:59:35.331645 28797 raft_consensus.cc:359] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.331897 28797 raft_consensus.cc:385] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.332010 28797 raft_consensus.cc:740] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.332309 28797 consensus_queue.cc:260] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.333042 28797 ts_tablet_manager.cc:1434] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:35.333384 28797 tablet_bootstrap.cc:492] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:35.334986 28989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.62:43001, user_credentials={real_user=slave}} blocked reactor thread for 45633.7us
I20260618 13:59:35.335927 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 60e02f51b8624a739fadbf4fafe90cb5 (DEFAULT_TABLE table=default.Table_84a1abff52a348629514e93e578bfac3 [id=db6c4f85b83849128ab1b1447e7357be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.336387 28529 tablet_service.cc:1511] Processing CreateTablet for tablet daface32f2a94a49a2db0729dfba32c2 (DEFAULT_TABLE table=default.Table_1a2d838221d44ca1952452c1475a0498 [id=abfc17f0ea554cccb80d20a8f534749d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.338871 28652 tablet_service.cc:1511] Processing CreateTablet for tablet 60e02f51b8624a739fadbf4fafe90cb5 (DEFAULT_TABLE table=default.Table_84a1abff52a348629514e93e578bfac3 [id=db6c4f85b83849128ab1b1447e7357be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.339262 28652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60e02f51b8624a739fadbf4fafe90cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.345487 28596 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:35.345526 29746 ts_tablet_manager.cc:1929] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:35.345656 29746 log.cc:1199] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/c84189a4b1194c87bbdf81f6e0ed56e4
I20260618 13:59:35.346043 29746 ts_tablet_manager.cc:1950] T c84189a4b1194c87bbdf81f6e0ed56e4 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
W20260618 13:59:35.346673 28617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 63578.7us
I20260618 13:59:35.349772 28797 tablet_bootstrap.cc:654] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:35.351796 29020 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 92) took 1992 ms. Trace:
I20260618 13:59:35.351897 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60e02f51b8624a739fadbf4fafe90cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.352731 29020 rpcz_store.cc:276] 0618 13:59:33.359419 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.039881 (+1680462us) service_pool.cc:224] Handling call
0618 13:59:35.190049 (+150168us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:35.191090 (+ 1041us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:35.351522 (+160432us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:35.351638 (+ 116us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:35.351749 (+ 111us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:35.193645 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:35.193765 (+ 120us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:35.193771 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:35.193774 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:35.193862 (+ 88us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:35.193874 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:35.193897 (+ 23us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:35.193972 (+ 75us) tablet.cc:701] Row locks acquired
0618 13:59:35.193974 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:35.194007 (+ 33us) write_op.cc:270] Start()
0618 13:59:35.194038 (+ 31us) write_op.cc:276] Timestamp: P: 1781791175193941 usec, L: 0
0618 13:59:35.194041 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:35.194275 (+ 234us) log.cc:844] Serialized 433 byte log entry
0618 13:59:35.342301 (+148026us) op_driver.cc:464] REPLICATION: finished
0618 13:59:35.349129 (+ 6828us) write_op.cc:301] APPLY: starting
0618 13:59:35.349140 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:35.349172 (+ 32us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:35.349219 (+ 47us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:35.349325 (+ 106us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:35.349353 (+ 28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:35.349360 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:35.349416 (+ 56us) log.cc:844] Serialized 29 byte log entry
0618 13:59:35.349460 (+ 44us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.349473 (+ 13us) write_op.cc:454] Released schema lock
0618 13:59:35.349480 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":6790,"apply.run_cpu_time_us":409,"apply.run_wall_time_us":408,"num_ops":2,"prepare.queue_time_us":50,"prepare.run_cpu_time_us":709,"prepare.run_wall_time_us":709,"replication_time_us":148240,"threads_started":1}]]}
I20260618 13:59:35.352268 29017 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 89) took 2139 ms. Trace:
I20260618 13:59:35.354717 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daface32f2a94a49a2db0729dfba32c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.353785 29017 rpcz_store.cc:276] 0618 13:59:33.213120 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.841277 (+1628157us) service_pool.cc:224] Handling call
0618 13:59:35.183007 (+341730us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:35.184018 (+ 1011us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:35.352056 (+168038us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:35.352154 (+ 98us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:35.352228 (+ 74us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:35.184757 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:35.184920 (+ 163us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:35.184926 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:35.184930 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:35.185048 (+ 118us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:35.185059 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:35.185063 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:35.185166 (+ 103us) tablet.cc:701] Row locks acquired
0618 13:59:35.185168 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:35.185203 (+ 35us) write_op.cc:270] Start()
0618 13:59:35.185260 (+ 57us) write_op.cc:276] Timestamp: P: 1781791175185137 usec, L: 0
0618 13:59:35.185264 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:35.185473 (+ 209us) log.cc:844] Serialized 433 byte log entry
0618 13:59:35.342207 (+156734us) op_driver.cc:464] REPLICATION: finished
0618 13:59:35.348500 (+ 6293us) write_op.cc:301] APPLY: starting
0618 13:59:35.348519 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:35.348570 (+ 51us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:35.348608 (+ 38us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:35.348721 (+ 113us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:35.348774 (+ 53us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:35.348780 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:35.348865 (+ 85us) log.cc:844] Serialized 29 byte log entry
0618 13:59:35.348937 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.348957 (+ 20us) write_op.cc:454] Released schema lock
0618 13:59:35.348963 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:35.349055 (+ 92us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.349059 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":6199,"apply.run_cpu_time_us":542,"apply.run_wall_time_us":542,"num_ops":2,"prepare.queue_time_us":64,"prepare.run_cpu_time_us":790,"prepare.run_wall_time_us":790,"replication_time_us":156906}]]}
I20260618 13:59:35.362702 28798 tablet_bootstrap.cc:492] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:35.368772 28798 tablet_bootstrap.cc:654] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:35.388437 28989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.62:43001, user_credentials={real_user=slave}} blocked reactor thread for 46579us
I20260618 13:59:35.401412 28540 raft_consensus.cc:3060] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.402400 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet d6871232860b41d489e34d95eb897241 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:35.404767 28618 consensus_peers.cc:597] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0fae928833e44d2ba860d6184d4c0142. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:35.406150 28383 tablet_service.cc:1511] Processing CreateTablet for tablet daface32f2a94a49a2db0729dfba32c2 (DEFAULT_TABLE table=default.Table_1a2d838221d44ca1952452c1475a0498 [id=abfc17f0ea554cccb80d20a8f534749d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.409775 28652 tablet_service.cc:1511] Processing CreateTablet for tablet daface32f2a94a49a2db0729dfba32c2 (DEFAULT_TABLE table=default.Table_1a2d838221d44ca1952452c1475a0498 [id=abfc17f0ea554cccb80d20a8f534749d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.410193 28652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daface32f2a94a49a2db0729dfba32c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.413218 29024 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet f45556a59c424ca2bc30956566374c89
I20260618 13:59:35.414536 29024 catalog_manager.cc:6784] ReplaceTablet: tablet f45556a59c424ca2bc30956566374c89 deleted and replaced by tablet 6903cb63c5b04408b7eaecf0c0231c42
I20260618 13:59:35.414784 29024 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 91) took 2106 ms. Trace:
I20260618 13:59:35.414963 29024 rpcz_store.cc:276] 0618 13:59:33.307818 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.980540 (+1672722us) service_pool.cc:224] Handling call
0618 13:59:35.414714 (+434174us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:35.327681 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:35.327787 (+ 106us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:35.327792 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:35.327795 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:35.327892 (+ 97us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:35.327902 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:35.327907 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:35.327982 (+ 75us) tablet.cc:701] Row locks acquired
0618 13:59:35.327984 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:35.328016 (+ 32us) write_op.cc:270] Start()
0618 13:59:35.328043 (+ 27us) write_op.cc:276] Timestamp: P: 1781791175327991 usec, L: 0
0618 13:59:35.328046 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:35.328209 (+ 163us) log.cc:844] Serialized 625 byte log entry
0618 13:59:35.411980 (+ 83771us) op_driver.cc:464] REPLICATION: finished
0618 13:59:35.412055 (+ 75us) write_op.cc:301] APPLY: starting
0618 13:59:35.412066 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:35.412096 (+ 30us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:35.412116 (+ 20us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:35.412280 (+ 164us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:35.412384 (+ 104us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:35.412391 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:35.412466 (+ 75us) log.cc:844] Serialized 29 byte log entry
0618 13:59:35.412531 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.412570 (+ 39us) write_op.cc:454] Released schema lock
0618 13:59:35.412575 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":35,"apply.run_cpu_time_us":611,"apply.run_wall_time_us":611,"num_ops":2,"prepare.queue_time_us":51,"prepare.run_cpu_time_us":591,"prepare.run_wall_time_us":590,"replication_time_us":83908}]]}
I20260618 13:59:35.416183 29024 master_service.cc:960] ReplaceTablet: received request to replace tablet 98446f40214c4ac984fcfe98310005c5 from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:35.416364 29746 tablet_replica.cc:333] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:35.416587 29746 raft_consensus.cc:2243] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:35.416801 29746 raft_consensus.cc:2272] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:35.417569 29023 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 95) took 1980 ms. Trace:
I20260618 13:59:35.418156 29566 ts_tablet_manager.cc:1929] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:35.418717 29566 log.cc:1199] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/5c75278db2234b11bf95f1cf5cc76edd
I20260618 13:59:35.417732 29023 rpcz_store.cc:276] 0618 13:59:33.436622 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.280300 (+1843678us) service_pool.cc:224] Handling call
0618 13:59:35.281826 (+ 1526us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:35.282389 (+ 563us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:35.417341 (+134952us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:35.417445 (+ 104us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:35.417533 (+ 88us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:35.297449 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:35.297566 (+ 117us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:35.297571 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:35.297574 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:35.297659 (+ 85us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:35.297670 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:35.297675 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:35.297810 (+ 135us) tablet.cc:701] Row locks acquired
0618 13:59:35.297815 (+ 5us) write_op.cc:260] PREPARE: finished
0618 13:59:35.297860 (+ 45us) write_op.cc:270] Start()
0618 13:59:35.297897 (+ 37us) write_op.cc:276] Timestamp: P: 1781791175297835 usec, L: 0
0618 13:59:35.297900 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:35.298092 (+ 192us) log.cc:844] Serialized 433 byte log entry
0618 13:59:35.409865 (+111773us) op_driver.cc:464] REPLICATION: finished
0618 13:59:35.409992 (+ 127us) write_op.cc:301] APPLY: starting
0618 13:59:35.410004 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:35.410048 (+ 44us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:35.410070 (+ 22us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:35.410169 (+ 99us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:35.410200 (+ 31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:35.410206 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:35.410292 (+ 86us) log.cc:844] Serialized 29 byte log entry
0618 13:59:35.410355 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.410371 (+ 16us) write_op.cc:454] Released schema lock
0618 13:59:35.410378 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:35.410480 (+ 102us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.410484 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":84,"apply.run_cpu_time_us":475,"apply.run_wall_time_us":474,"num_ops":2,"prepare.queue_time_us":57,"prepare.run_cpu_time_us":716,"prepare.run_wall_time_us":716,"replication_time_us":111931}]]}
I20260618 13:59:35.419102 29566 ts_tablet_manager.cc:1950] T 5c75278db2234b11bf95f1cf5cc76edd P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:35.419423 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 60) took 3373 ms. Trace:
I20260618 13:59:35.419499 29566 rpcz_store.cc:276] 0618 13:59:32.045834 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.045892 (+ 58us) service_pool.cc:224] Handling call
0618 13:59:35.216587 (+3170695us) ts_tablet_manager.cc:1151] Deleting tablet 5c75278db2234b11bf95f1cf5cc76edd
0618 13:59:35.216595 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:35.316545 (+ 99950us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:35.418101 (+101556us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:35.419406 (+ 1305us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3170521}
I20260618 13:59:35.419808 29566 tablet_replica.cc:333] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:35.419945 29023 master_service.cc:960] ReplaceTablet: received request to replace tablet 3dd0eaa64271489d92aa380df877f257 from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:35.420004 29566 raft_consensus.cc:2243] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:35.420485 29749 raft_consensus.cc:493] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:35.420660 29749 raft_consensus.cc:515] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.421290 29749 leader_election.cc:290] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:35.420320 29566 raft_consensus.cc:2272] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:35.422340 28484 leader_election.cc:343] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: d9c00003300246ada2a356e3e1cf684b
I20260618 13:59:35.423100 29026 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 94) took 2044 ms. Trace:
W20260618 13:59:35.424726 28484 consensus_peers.cc:597] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 -> Peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Couldn't send request to peer d6a169b16ced46ca9cfbc631709bb257. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eead369de58f40378254697cbbbc6958. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:35.423259 29026 rpcz_store.cc:276] 0618 13:59:33.378938 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.277175 (+1898237us) service_pool.cc:224] Handling call
0618 13:59:35.278564 (+ 1389us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:35.279124 (+ 560us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:35.420257 (+141133us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:35.422951 (+ 2694us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:35.423061 (+ 110us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:35.314726 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:35.314862 (+ 136us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:35.314869 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:35.314871 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:35.314961 (+ 90us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:35.314972 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:35.314976 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:35.315052 (+ 76us) tablet.cc:701] Row locks acquired
0618 13:59:35.315054 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:35.315088 (+ 34us) write_op.cc:270] Start()
0618 13:59:35.315119 (+ 31us) write_op.cc:276] Timestamp: P: 1781791175315064 usec, L: 0
0618 13:59:35.315122 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:35.315351 (+ 229us) log.cc:844] Serialized 433 byte log entry
0618 13:59:35.409933 (+ 94582us) op_driver.cc:464] REPLICATION: finished
0618 13:59:35.410550 (+ 617us) write_op.cc:301] APPLY: starting
0618 13:59:35.410560 (+ 10us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:35.410591 (+ 31us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:35.410624 (+ 33us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:35.410707 (+ 83us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:35.410733 (+ 26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:35.410739 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:35.410804 (+ 65us) log.cc:844] Serialized 29 byte log entry
0618 13:59:35.410844 (+ 40us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.410856 (+ 12us) write_op.cc:454] Released schema lock
0618 13:59:35.410862 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":586,"apply.run_cpu_time_us":366,"apply.run_wall_time_us":365,"num_ops":2,"prepare.queue_time_us":55,"prepare.run_cpu_time_us":700,"prepare.run_wall_time_us":700,"replication_time_us":94796}]]}
I20260618 13:59:35.426292 28798 tablet_bootstrap.cc:492] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:35.426522 28798 ts_tablet_manager.cc:1403] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.064s user 0.003s sys 0.002s
I20260618 13:59:35.426906 29018 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 93) took 2067 ms. Trace:
I20260618 13:59:35.427065 29018 rpcz_store.cc:276] 0618 13:59:33.359537 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.111813 (+1752276us) service_pool.cc:224] Handling call
0618 13:59:35.275838 (+164025us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:35.276438 (+ 600us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:35.426530 (+150092us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:35.426699 (+ 169us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:35.426816 (+ 117us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:35.317949 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:35.318048 (+ 99us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:35.318054 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:35.318057 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:35.318131 (+ 74us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:35.318139 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:35.318143 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:35.318217 (+ 74us) tablet.cc:701] Row locks acquired
0618 13:59:35.318219 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:35.318252 (+ 33us) write_op.cc:270] Start()
0618 13:59:35.318281 (+ 29us) write_op.cc:276] Timestamp: P: 1781791175318228 usec, L: 0
0618 13:59:35.318285 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:35.318488 (+ 203us) log.cc:844] Serialized 433 byte log entry
0618 13:59:35.411257 (+ 92769us) op_driver.cc:464] REPLICATION: finished
0618 13:59:35.411354 (+ 97us) write_op.cc:301] APPLY: starting
0618 13:59:35.411366 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:35.411395 (+ 29us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:35.411423 (+ 28us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:35.411508 (+ 85us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:35.411533 (+ 25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:35.411540 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:35.411590 (+ 50us) log.cc:844] Serialized 29 byte log entry
0618 13:59:35.411640 (+ 50us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.411652 (+ 12us) write_op.cc:454] Released schema lock
0618 13:59:35.411658 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":54,"apply.run_cpu_time_us":355,"apply.run_wall_time_us":355,"num_ops":2,"prepare.queue_time_us":50,"prepare.run_cpu_time_us":603,"prepare.run_wall_time_us":603,"replication_time_us":92948}]]}
I20260618 13:59:35.428678 28798 raft_consensus.cc:359] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.429121 28798 raft_consensus.cc:385] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.429292 28798 raft_consensus.cc:740] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
W20260618 13:59:35.430402 28483 leader_election.cc:343] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: d9c00003300246ada2a356e3e1cf684b
I20260618 13:59:35.430564 28483 leader_election.cc:304] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:35.430979 29749 raft_consensus.cc:2749] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:35.431531 29746 ts_tablet_manager.cc:1916] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:35.433857 29026 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_e372385b751446e28fe8798e387b2148" } modify_external_catalogs: true
I20260618 13:59:35.434121 29026 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_e372385b751446e28fe8798e387b2148" } modify_external_catalogs: true
I20260618 13:59:35.435181 29017 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_795f17a0755242fba8eb1a55877bd470"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:35.435824 29017 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_795f17a0755242fba8eb1a55877bd470 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:35.440167 28458 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:35.440579 29018 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_7e5d58ad3e8043919b66e8890819535c" } modify_external_catalogs: true
I20260618 13:59:35.441905 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet f45556a59c424ca2bc30956566374c89 with delete_type TABLET_DATA_DELETED (replaced by tablet 6903cb63c5b04408b7eaecf0c0231c42) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.442003 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet f45556a59c424ca2bc30956566374c89 with delete_type TABLET_DATA_DELETED (replaced by tablet 6903cb63c5b04408b7eaecf0c0231c42) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:35.445330 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 5c75278db2234b11bf95f1cf5cc76edd (table default.Table_9d7999641cc344f18ccc672c49518192 [id=6ac90f0f8ac84ed5953d37156d9b768b]) successfully deleted
I20260618 13:59:35.445585 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet c84189a4b1194c87bbdf81f6e0ed56e4 (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]) successfully deleted
I20260618 13:59:35.448649 29914 raft_consensus.cc:493] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.449290 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet f45556a59c424ca2bc30956566374c89 with delete_type TABLET_DATA_DELETED (replaced by tablet 6903cb63c5b04408b7eaecf0c0231c42) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:35.450344 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daface32f2a94a49a2db0729dfba32c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.452369 28799 tablet_bootstrap.cc:654] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:35.458956 29914 raft_consensus.cc:515] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.460973 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet eead369de58f40378254697cbbbc6958 with delete_type TABLET_DATA_DELETED (replaced by tablet d464dd402d2044469e364ed37fdfb39b) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:35.461597 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c75278db2234b11bf95f1cf5cc76edd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:35.468416 29914 leader_election.cc:290] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:35.469729 29714 consensus_queue.cc:1048] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:35.470177 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c38a2156a55e4e5eb91d101e9a61eecd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:35.470374 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c38a2156a55e4e5eb91d101e9a61eecd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:35.471669 29714 consensus_queue.cc:1048] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:35.475210 28346 leader_election.cc:343] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:35.475492 28346 leader_election.cc:343] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.477641 28346 leader_election.cc:304] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:35.478917 29714 raft_consensus.cc:493] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.479058 29714 raft_consensus.cc:515] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.479660 29714 leader_election.cc:290] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.480337 29914 raft_consensus.cc:2749] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:35.481420 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "538077e459a44ed78c95f2e881b6e0b0" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:35.482321 28618 leader_election.cc:343] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.482764 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "538077e459a44ed78c95f2e881b6e0b0" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:35.483614 28799 tablet_bootstrap.cc:492] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:35.483618 28409 raft_consensus.cc:2468] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:35.484210 28799 ts_tablet_manager.cc:1403] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.154s user 0.001s sys 0.007s
I20260618 13:59:35.484339 28617 leader_election.cc:304] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.484607 29714 raft_consensus.cc:2804] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:35.484683 29714 raft_consensus.cc:493] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:35.484743 29714 raft_consensus.cc:3060] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.487598 29714 raft_consensus.cc:515] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.488970 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "538077e459a44ed78c95f2e881b6e0b0" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:35.488719 29714 leader_election.cc:290] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:35.489521 28618 leader_election.cc:343] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.490346 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "538077e459a44ed78c95f2e881b6e0b0" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:35.491443 28409 raft_consensus.cc:3060] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.492458 28799 raft_consensus.cc:359] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.492663 28799 raft_consensus.cc:385] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.492744 28799 raft_consensus.cc:740] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.494686 28409 raft_consensus.cc:2468] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:35.495169 28617 leader_election.cc:304] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.494374 28799 consensus_queue.cc:260] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.495700 29714 raft_consensus.cc:2804] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:35.495868 29714 raft_consensus.cc:697] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:35.496158 29714 consensus_queue.cc:237] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.496742 28799 ts_tablet_manager.cc:1434] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20260618 13:59:35.499277 28797 tablet_bootstrap.cc:492] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:35.499459 28797 ts_tablet_manager.cc:1403] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.166s user 0.021s sys 0.025s
I20260618 13:59:35.500288 28797 raft_consensus.cc:359] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.500530 28797 raft_consensus.cc:385] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.500635 28797 raft_consensus.cc:740] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.500876 28797 consensus_queue.cc:260] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.501993 28797 ts_tablet_manager.cc:1434] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:35.502445 28799 tablet_bootstrap.cc:492] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:35.503300 28797 tablet_bootstrap.cc:492] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:35.504138 28799 tablet_bootstrap.cc:654] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:35.510478 28618 consensus_peers.cc:597] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:35.523626 30059 raft_consensus.cc:493] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.523788 30059 raft_consensus.cc:515] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.524317 30059 leader_election.cc:290] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.525352 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab521028ac1a4d0da013112d76bb169f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:35.525624 28407 raft_consensus.cc:2468] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:35.526552 28798 consensus_queue.cc:260] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.530371 30059 consensus_queue.cc:1048] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:35.539270 28798 ts_tablet_manager.cc:1434] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.113s user 0.002s sys 0.000s
W20260618 13:59:35.540601 29011 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.3:42065 (request call id 143) took 2093 ms (client timeout 1000 ms). Trace:
I20260618 13:59:35.542495 29566 ts_tablet_manager.cc:1916] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:35.545239 29914 raft_consensus.cc:493] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.545385 29914 raft_consensus.cc:515] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.546027 29914 leader_election.cc:290] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.544332 28797 tablet_bootstrap.cc:654] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:35.546736 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f7eaa7d218440ca2c35340c5c87814" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:35.549463 28346 leader_election.cc:343] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.546598 28798 tablet_bootstrap.cc:492] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
W20260618 13:59:35.542133 29011 rpcz_store.cc:269] 0618 13:59:33.446122 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:33.642029 (+195907us) service_pool.cc:224] Handling call
0618 13:59:35.539925 (+1897896us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:34.233115 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:34.233631 (+ 516us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:34.233638 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:34.233641 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:34.233821 (+ 180us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:34.233835 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:34.233840 (+ 5us) tablet.cc:685] Acquiring locks for 5 operations
0618 13:59:34.233984 (+ 144us) tablet.cc:701] Row locks acquired
0618 13:59:34.233986 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:34.234024 (+ 38us) write_op.cc:270] Start()
0618 13:59:34.234057 (+ 33us) write_op.cc:276] Timestamp: P: 1781791174234007 usec, L: 0
0618 13:59:34.234061 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:34.234236 (+ 175us) log.cc:844] Serialized 2039 byte log entry
0618 13:59:34.241003 (+ 6767us) op_driver.cc:464] REPLICATION: finished
0618 13:59:34.241114 (+ 111us) write_op.cc:301] APPLY: starting
0618 13:59:34.241130 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:34.241188 (+ 58us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:34.241216 (+ 28us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:34.241353 (+ 137us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:34.241417 (+ 64us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0618 13:59:34.241429 (+ 12us) write_op.cc:312] APPLY: finished
0618 13:59:34.241538 (+ 109us) log.cc:844] Serialized 47 byte log entry
0618 13:59:34.241612 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.241644 (+ 32us) write_op.cc:454] Released schema lock
0618 13:59:34.241654 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:34.241740 (+ 86us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:34.241747 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":175747,"reported_tablets":198,"child_traces":[["op",{"apply.queue_time_us":60,"apply.run_cpu_time_us":611,"apply.run_wall_time_us":609,"num_ops":5,"prepare.queue_time_us":54,"prepare.run_cpu_time_us":888,"prepare.run_wall_time_us":1887,"raft.queue_time_us":38,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":222,"replication_time_us":6912}]]}
I20260618 13:59:35.547670 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f7eaa7d218440ca2c35340c5c87814" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:35.551146 28346 leader_election.cc:343] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.551246 28346 leader_election.cc:304] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:35.551903 29914 raft_consensus.cc:2749] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:35.555493 28618 consensus_peers.cc:597] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:35.541326 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab521028ac1a4d0da013112d76bb169f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:35.557782 29025 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 96) took 2088 ms. Trace:
W20260618 13:59:35.558118 28618 leader_election.cc:343] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.557988 29025 rpcz_store.cc:276] 0618 13:59:33.469321 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.306829 (+1837508us) service_pool.cc:224] Handling call
0618 13:59:35.307940 (+ 1111us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:35.308433 (+ 493us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:35.451850 (+143417us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:35.557636 (+105786us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:35.557741 (+ 105us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:35.372735 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:35.372859 (+ 124us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:35.372865 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:35.372867 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:35.372949 (+ 82us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:35.372959 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:35.372964 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:35.373035 (+ 71us) tablet.cc:701] Row locks acquired
0618 13:59:35.373037 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:35.373072 (+ 35us) write_op.cc:270] Start()
0618 13:59:35.373104 (+ 32us) write_op.cc:276] Timestamp: P: 1781791175373042 usec, L: 0
0618 13:59:35.373107 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:35.434723 (+ 61616us) log.cc:844] Serialized 433 byte log entry
0618 13:59:35.434914 (+ 191us) spinlock_profiling.cc:243] Waited 61.4 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fcbac75ef22 00007fcbadb40305 00007fcbadb3ee29 00007fcbadb47250 00007fcbadb4705d 0000000000aa681c 00007fcba07299ff 00007fcba072cd1b 00007fcba072cb98 0000000000aa681c
0618 13:59:35.447974 (+ 13060us) op_driver.cc:464] REPLICATION: finished
0618 13:59:35.450828 (+ 2854us) write_op.cc:301] APPLY: starting
0618 13:59:35.450847 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:35.450899 (+ 52us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:35.450904 (+ 5us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:35.451057 (+ 153us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:35.451284 (+ 227us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:35.451293 (+ 9us) write_op.cc:312] APPLY: finished
0618 13:59:35.451400 (+ 107us) log.cc:844] Serialized 29 byte log entry
0618 13:59:35.451642 (+ 242us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.451660 (+ 18us) write_op.cc:454] Released schema lock
0618 13:59:35.451667 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:35.451735 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.451739 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2723,"apply.run_cpu_time_us":560,"apply.run_wall_time_us":910,"num_ops":2,"prepare.queue_time_us":187,"prepare.run_cpu_time_us":1639,"prepare.run_wall_time_us":62252,"raft.queue_time_us":1058,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":265,"replication_time_us":74827,"spinlock_wait_cycles":141196544}]]}
I20260618 13:59:35.559968 29025 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_152a483501b54a67bbdc0cae4959d452" } modify_external_catalogs: true
I20260618 13:59:35.560217 29025 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_152a483501b54a67bbdc0cae4959d452" } modify_external_catalogs: true
I20260618 13:59:35.560905 29018 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_7e5d58ad3e8043919b66e8890819535c" } modify_external_catalogs: true
I20260618 13:59:35.569824 29017 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 97) took 2100 ms. Trace:
I20260618 13:59:35.570079 29017 rpcz_store.cc:276] 0618 13:59:33.469504 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.357831 (+1888327us) service_pool.cc:224] Handling call
0618 13:59:35.435119 (+ 77288us) spinlock_profiling.cc:243] Waited 76.5 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb46846c3 00007fcbab0e07c6 00007fcbab0e04f5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:35.557539 (+122420us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:35.558378 (+ 839us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:35.569524 (+ 11146us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:35.569654 (+ 130us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:35.569766 (+ 112us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:35.558978 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:35.559081 (+ 103us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:35.559087 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:35.559089 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:35.559170 (+ 81us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:35.559180 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:35.559184 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:35.559253 (+ 69us) tablet.cc:701] Row locks acquired
0618 13:59:35.559255 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:35.559291 (+ 36us) write_op.cc:270] Start()
0618 13:59:35.559320 (+ 29us) write_op.cc:276] Timestamp: P: 1781791175559240 usec, L: 0
0618 13:59:35.559324 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:35.559504 (+ 180us) log.cc:844] Serialized 433 byte log entry
0618 13:59:35.567042 (+ 7538us) op_driver.cc:464] REPLICATION: finished
0618 13:59:35.567431 (+ 389us) write_op.cc:301] APPLY: starting
0618 13:59:35.568813 (+ 1382us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:35.568858 (+ 45us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:35.568905 (+ 47us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:35.569000 (+ 95us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:35.569030 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:35.569038 (+ 8us) write_op.cc:312] APPLY: finished
0618 13:59:35.569105 (+ 67us) log.cc:844] Serialized 29 byte log entry
0618 13:59:35.569165 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.569183 (+ 18us) write_op.cc:454] Released schema lock
0618 13:59:35.569189 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:35.569259 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.569263 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":175967488,"child_traces":[["op",{"apply.queue_time_us":313,"apply.run_cpu_time_us":480,"apply.run_wall_time_us":1836,"num_ops":2,"prepare.queue_time_us":52,"prepare.run_cpu_time_us":631,"prepare.run_wall_time_us":631,"raft.queue_time_us":75,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":240,"replication_time_us":7688}]]}
I20260618 13:59:35.580448 28798 tablet_bootstrap.cc:654] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:35.581493 29017 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_7f787ab851be496dae97cba8835db19b" } new_table_name: "default.Table_0ac51b9af8d041798d81691aac77af3a" modify_external_catalogs: true
I20260618 13:59:35.585901 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:35.586176 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:35.586781 29745 ts_tablet_manager.cc:1929] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:35.587054 29745 log.cc:1199] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/41107eb1d9164a9c98060003b4027412
I20260618 13:59:35.587723 29745 ts_tablet_manager.cc:1950] T 41107eb1d9164a9c98060003b4027412 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:35.588254 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 58) took 2682 ms. Trace:
I20260618 13:59:35.588424 29745 rpcz_store.cc:276] 0618 13:59:32.906037 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.907800 (+ 1763us) service_pool.cc:224] Handling call
0618 13:59:35.262657 (+2354857us) ts_tablet_manager.cc:1151] Deleting tablet 41107eb1d9164a9c98060003b4027412
0618 13:59:35.262683 (+ 26us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:35.452021 (+189338us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:35.586730 (+134709us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:35.588218 (+ 1488us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2354727}
I20260618 13:59:35.589073 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 41107eb1d9164a9c98060003b4027412 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]) successfully deleted
I20260618 13:59:35.589236 29745 tablet_replica.cc:333] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:35.589769 29745 raft_consensus.cc:2243] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:35.590216 29745 raft_consensus.cc:2272] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:35.590584 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:35.590996 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:35.591228 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:35.591420 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:35.591574 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:35.591784 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_c70d6c5b0dc94b33b4c5772edd2b9dca [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 2 got 0
I20260618 13:59:35.597322 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:35.597575 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c881dd3cb66344969310956ad6e83cc0 (table default.Table_0158779f563348c78b3505f98d43b839 [id=b1d85ea3aa7047d285ab7c86da8213fd]). Expected version 1 got 0
I20260618 13:59:35.599594 29746 ts_tablet_manager.cc:1929] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:35.599653 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:35.600461 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:35.600713 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:35.601020 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_7f787ab851be496dae97cba8835db19b [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 1 got 0
I20260618 13:59:35.601212 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:35.601565 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:35.601774 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 15180aefba0544329daee74308ef5010 (table default.Table_97b6bb39b3bf4612be391b9dd78d1646 [id=42a2a24590834abc9537669aaf223a7c]). Expected version 1 got 0
I20260618 13:59:35.602471 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:35.602701 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:35.602885 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:35.607771 29914 raft_consensus.cc:493] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.607913 29914 raft_consensus.cc:515] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.608537 29914 leader_election.cc:290] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.610025 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1968e5ce2e42a5b836c90b0904f158" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:35.610532 28346 leader_election.cc:343] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.610859 28797 tablet_bootstrap.cc:492] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:35.611014 28797 ts_tablet_manager.cc:1403] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.108s user 0.007s sys 0.003s
I20260618 13:59:35.611735 28797 raft_consensus.cc:359] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.611912 28797 raft_consensus.cc:385] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.611976 28797 raft_consensus.cc:740] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.612179 28797 consensus_queue.cc:260] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.612709 28797 ts_tablet_manager.cc:1434] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:35.612743 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:35.613014 28797 tablet_bootstrap.cc:492] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:35.613452 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:35.613824 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:35.614245 28617 leader_election.cc:304] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.614626 29303 raft_consensus.cc:2804] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:35.614722 29303 raft_consensus.cc:493] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:35.614818 29303 raft_consensus.cc:3060] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.615144 28797 tablet_bootstrap.cc:654] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:35.616330 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:35.616539 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:35.617182 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 72c04f55890648f0a8184f3783243f7c (table default.Table_773be314d7e741fcb4fb6eb7d954802e [id=0d83fc68255c458587390c58b12e00d7]). Expected version 2 got 0
I20260618 13:59:35.617352 29303 raft_consensus.cc:515] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.618290 29303 leader_election.cc:290] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.619243 28799 tablet_bootstrap.cc:492] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:35.619437 28799 ts_tablet_manager.cc:1403] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.117s user 0.000s sys 0.004s
I20260618 13:59:35.620285 28799 raft_consensus.cc:359] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.620513 28799 raft_consensus.cc:385] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.620620 28799 raft_consensus.cc:740] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.620571 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab521028ac1a4d0da013112d76bb169f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:35.620879 28403 raft_consensus.cc:3060] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.623721 28403 raft_consensus.cc:2468] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:35.625557 28617 leader_election.cc:304] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:35.625952 29303 raft_consensus.cc:2804] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:35.626189 29303 raft_consensus.cc:697] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:35.626458 29303 consensus_queue.cc:237] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.629751 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:35.630033 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a]). Expected version 1 got 0
I20260618 13:59:35.630255 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 39b06ae7444a48dbbec9e096fd7e4052 (table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15]). Expected version 2 got 0
I20260618 13:59:35.634753 28797 tablet_bootstrap.cc:492] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:35.634925 28797 ts_tablet_manager.cc:1403] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.003s
I20260618 13:59:35.635726 28797 raft_consensus.cc:359] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.635919 28797 raft_consensus.cc:385] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.635995 28797 raft_consensus.cc:740] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.636248 28797 consensus_queue.cc:260] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.636926 28797 ts_tablet_manager.cc:1434] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:35.637312 28797 tablet_bootstrap.cc:492] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:35.620885 28799 consensus_queue.cc:260] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.638051 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:35.638324 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:35.639153 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:35.640202 28799 ts_tablet_manager.cc:1434] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.021s user 0.001s sys 0.001s
I20260618 13:59:35.642128 29914 raft_consensus.cc:493] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.642361 29914 raft_consensus.cc:515] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.643033 29914 leader_election.cc:290] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.644017 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fa1a27fc9304e18b96cad9f923779f2" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:35.644534 28346 leader_election.cc:343] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.645004 28799 tablet_bootstrap.cc:492] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:35.647910 28799 tablet_bootstrap.cc:654] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:35.609740 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1968e5ce2e42a5b836c90b0904f158" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:35.619848 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab521028ac1a4d0da013112d76bb169f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:35.643865 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fa1a27fc9304e18b96cad9f923779f2" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:35.651551 28346 leader_election.cc:343] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.651654 28346 leader_election.cc:304] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:35.652107 29914 raft_consensus.cc:2749] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:35.652585 28346 leader_election.cc:343] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.652680 28346 leader_election.cc:304] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:35.653081 29914 raft_consensus.cc:2749] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:35.599848 29746 log.cc:1199] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/b51d0851efb34eaa9e9e4be6b292ef62
W20260618 13:59:35.653528 28618 leader_election.cc:343] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.653927 29746 ts_tablet_manager.cc:1950] T b51d0851efb34eaa9e9e4be6b292ef62 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:35.654512 28797 tablet_bootstrap.cc:654] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:35.654628 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 80) took 1097 ms. Trace:
I20260618 13:59:35.654834 29746 rpcz_store.cc:276] 0618 13:59:34.557030 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.557085 (+ 55us) service_pool.cc:224] Handling call
0618 13:59:35.416278 (+859193us) ts_tablet_manager.cc:1151] Deleting tablet b51d0851efb34eaa9e9e4be6b292ef62
0618 13:59:35.416289 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:35.543851 (+127562us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:35.599473 (+ 55622us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:35.654548 (+ 55075us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":859002}
I20260618 13:59:35.655670 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet b51d0851efb34eaa9e9e4be6b292ef62 (table default.Table_7d317fa28e3142a19d15c8360151c357 [id=bba3ab8821ca48a5aa1104094d78892a]) successfully deleted
I20260618 13:59:35.656160 29746 tablet_replica.cc:333] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:35.656575 29746 raft_consensus.cc:2243] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:35.656966 29746 raft_consensus.cc:2272] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:35.658069 29746 ts_tablet_manager.cc:1916] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:35.663463 29023 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 99) took 1780 ms. Trace:
I20260618 13:59:35.663573 29023 rpcz_store.cc:276] 0618 13:59:33.882569 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.419830 (+1537261us) service_pool.cc:224] Handling call
0618 13:59:35.439543 (+ 19713us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb469154b 00007fcbab0ed107 00007fcbab0ece35 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:35.663406 (+223863us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39577472}
I20260618 13:59:35.667286 28799 tablet_bootstrap.cc:492] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:35.667433 28799 ts_tablet_manager.cc:1403] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.004s
I20260618 13:59:35.668201 28799 raft_consensus.cc:359] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.668365 28799 raft_consensus.cc:385] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.668418 28799 raft_consensus.cc:740] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.668609 28799 consensus_queue.cc:260] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.671084 28799 ts_tablet_manager.cc:1434] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 13:59:35.671417 28799 tablet_bootstrap.cc:492] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:35.674175 28799 tablet_bootstrap.cc:654] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:35.687493 28799 tablet_bootstrap.cc:492] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:35.687701 28799 ts_tablet_manager.cc:1403] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.003s
I20260618 13:59:35.688436 28799 raft_consensus.cc:359] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.688659 28799 raft_consensus.cc:385] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.688742 28799 raft_consensus.cc:740] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.689011 28799 consensus_queue.cc:260] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.689630 28799 ts_tablet_manager.cc:1434] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:35.689944 28799 tablet_bootstrap.cc:492] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:35.691388 28799 tablet_bootstrap.cc:654] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:35.693565 29024 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 98446f40214c4ac984fcfe98310005c5
I20260618 13:59:35.694134 29024 catalog_manager.cc:6784] ReplaceTablet: tablet 98446f40214c4ac984fcfe98310005c5 deleted and replaced by tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3
I20260618 13:59:35.694257 29024 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 98) took 1929 ms. Trace:
I20260618 13:59:35.694363 29024 rpcz_store.cc:276] 0618 13:59:33.764613 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.416097 (+1651484us) service_pool.cc:224] Handling call
0618 13:59:35.433568 (+ 17471us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb469154b 00007fcbab0ed107 00007fcbab0ece35 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:35.694216 (+260648us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:35.685505 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:35.685609 (+ 104us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:35.685614 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:35.685617 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:35.685718 (+ 101us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:35.685727 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:35.685731 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:35.685812 (+ 81us) tablet.cc:701] Row locks acquired
0618 13:59:35.685815 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:35.685845 (+ 30us) write_op.cc:270] Start()
0618 13:59:35.685901 (+ 56us) write_op.cc:276] Timestamp: P: 1781791175685783 usec, L: 0
0618 13:59:35.685905 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:35.686068 (+ 163us) log.cc:844] Serialized 625 byte log entry
0618 13:59:35.690821 (+ 4753us) op_driver.cc:464] REPLICATION: finished
0618 13:59:35.692680 (+ 1859us) write_op.cc:301] APPLY: starting
0618 13:59:35.692698 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:35.692740 (+ 42us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:35.692780 (+ 40us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:35.692924 (+ 144us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:35.692981 (+ 57us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:35.692991 (+ 10us) write_op.cc:312] APPLY: finished
0618 13:59:35.693112 (+ 121us) log.cc:844] Serialized 29 byte log entry
0618 13:59:35.693174 (+ 62us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.693194 (+ 20us) write_op.cc:454] Released schema lock
0618 13:59:35.693203 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:35.693278 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.693283 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":39219968,"child_traces":[["op",{"apply.queue_time_us":1800,"apply.run_cpu_time_us":561,"apply.run_wall_time_us":593,"num_ops":2,"prepare.queue_time_us":4486,"prepare.run_cpu_time_us":636,"prepare.run_wall_time_us":636,"replication_time_us":4892}]]}
I20260618 13:59:35.695232 29024 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_7fa3da42543d4978aac5ac383ed10056" } new_table_name: "default.Table_55e47ed77544480a87dcb22d905c4418" modify_external_catalogs: true
I20260618 13:59:35.699543 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet 98446f40214c4ac984fcfe98310005c5 with delete_type TABLET_DATA_DELETED (replaced by tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:35.696584 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet 98446f40214c4ac984fcfe98310005c5 with delete_type TABLET_DATA_DELETED (replaced by tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:35.700150 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 98446f40214c4ac984fcfe98310005c5 with delete_type TABLET_DATA_DELETED (replaced by tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:35.710220 28799 tablet_bootstrap.cc:492] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:35.710381 28799 ts_tablet_manager.cc:1403] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.006s
I20260618 13:59:35.711122 28799 raft_consensus.cc:359] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.711434 28799 raft_consensus.cc:385] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.711501 28799 raft_consensus.cc:740] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.711742 28799 consensus_queue.cc:260] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.712347 28799 ts_tablet_manager.cc:1434] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:35.712622 28799 tablet_bootstrap.cc:492] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:35.714493 28799 tablet_bootstrap.cc:654] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:35.671098 29745 ts_tablet_manager.cc:1916] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:35.727571 28797 tablet_bootstrap.cc:492] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:35.727741 28797 ts_tablet_manager.cc:1403] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.091s user 0.001s sys 0.002s
I20260618 13:59:35.728811 28797 raft_consensus.cc:359] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.729012 28797 raft_consensus.cc:385] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.729087 28797 raft_consensus.cc:740] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.731019 28797 consensus_queue.cc:260] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.731673 28797 ts_tablet_manager.cc:1434] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 13:59:35.731976 28797 tablet_bootstrap.cc:492] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:35.735504 30060 raft_consensus.cc:493] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.735643 30060 raft_consensus.cc:515] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.736383 30060 leader_election.cc:290] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:35.736820 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d464dd402d2044469e364ed37fdfb39b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:35.736883 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d464dd402d2044469e364ed37fdfb39b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:35.737039 28407 raft_consensus.cc:2468] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:35.737330 28618 leader_election.cc:343] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.737851 28617 leader_election.cc:304] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.738402 29714 raft_consensus.cc:2804] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:35.738555 29714 raft_consensus.cc:493] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:35.738652 29714 raft_consensus.cc:3060] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.741787 29714 raft_consensus.cc:515] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.743053 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d464dd402d2044469e364ed37fdfb39b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:35.743569 28618 leader_election.cc:343] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.743853 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d464dd402d2044469e364ed37fdfb39b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:35.744060 28403 raft_consensus.cc:3060] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.747334 28403 raft_consensus.cc:2468] T d464dd402d2044469e364ed37fdfb39b P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:35.747850 28617 leader_election.cc:304] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.748306 29303 raft_consensus.cc:2804] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:35.748447 29303 raft_consensus.cc:697] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:35.748689 29303 consensus_queue.cc:237] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.749185 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 7d43224c621a4774870d44f0237b514a (DEFAULT_TABLE table=default.Table_995e1233361c4c2aae5e3ad5c9c989a9 [id=546662391a5d4e5e84824bd3947de2a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.749639 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d43224c621a4774870d44f0237b514a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.753032 28797 tablet_bootstrap.cc:654] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:35.753777 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 7d43224c621a4774870d44f0237b514a (DEFAULT_TABLE table=default.Table_995e1233361c4c2aae5e3ad5c9c989a9 [id=546662391a5d4e5e84824bd3947de2a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.754627 28652 tablet_service.cc:1511] Processing CreateTablet for tablet 7d43224c621a4774870d44f0237b514a (DEFAULT_TABLE table=default.Table_995e1233361c4c2aae5e3ad5c9c989a9 [id=546662391a5d4e5e84824bd3947de2a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.754971 28652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d43224c621a4774870d44f0237b514a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.764267 28529 tablet_service.cc:1511] Processing CreateTablet for tablet 5cbfebfca436461ab63dfe469ae14b4c (DEFAULT_TABLE table=default.Table_e9f8c36af8a047f09b3fa4cf54456229 [id=06c5b9a9c69348dbb2d0781f013ec4d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.765352 28657 tablet_service.cc:1511] Processing CreateTablet for tablet 5cbfebfca436461ab63dfe469ae14b4c (DEFAULT_TABLE table=default.Table_e9f8c36af8a047f09b3fa4cf54456229 [id=06c5b9a9c69348dbb2d0781f013ec4d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.765848 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cbfebfca436461ab63dfe469ae14b4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.770551 28384 tablet_service.cc:1511] Processing CreateTablet for tablet 5cbfebfca436461ab63dfe469ae14b4c (DEFAULT_TABLE table=default.Table_e9f8c36af8a047f09b3fa4cf54456229 [id=06c5b9a9c69348dbb2d0781f013ec4d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.770953 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cbfebfca436461ab63dfe469ae14b4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.770993 28386 tablet_service.cc:1511] Processing CreateTablet for tablet 4a9a5f630654465fb416f53af1424e9b (DEFAULT_TABLE table=default.Table_642894fa43114390bed040b60b648d86 [id=b0963be8750947539a7cbadf5f0cb582]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.771311 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a9a5f630654465fb416f53af1424e9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.771399 28798 tablet_bootstrap.cc:492] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:35.771539 28798 ts_tablet_manager.cc:1403] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.225s user 0.004s sys 0.000s
I20260618 13:59:35.772300 28798 raft_consensus.cc:359] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.772473 28798 raft_consensus.cc:385] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.772549 28798 raft_consensus.cc:740] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.774585 28798 consensus_queue.cc:260] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.774660 28390 tablet_service.cc:1511] Processing CreateTablet for tablet 60df0bb1a20043049275fdc25084dc0d (DEFAULT_TABLE table=default.Table_0a233b0cc52a4af38b49f5b81dbf08d0 [id=2ae931fac0214533ac148c97afdc69d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.775028 28390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60df0bb1a20043049275fdc25084dc0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.778713 28377 tablet_service.cc:1511] Processing CreateTablet for tablet d7c6c17ba06543219259749d7d38832d (DEFAULT_TABLE table=default.Table_11b11bd80db34f639f805447f8027a27 [id=be63cc607ee04972853447c9f771c131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.779083 28377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7c6c17ba06543219259749d7d38832d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.782673 28374 tablet_service.cc:1511] Processing CreateTablet for tablet 2128dbec33db453ba51070767a15b21e (DEFAULT_TABLE table=default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.790901 28798 ts_tablet_manager.cc:1434] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.019s user 0.002s sys 0.000s
I20260618 13:59:35.791347 28798 tablet_bootstrap.cc:492] T 7a9bd8577f4f4a9593a3ffeab582b02e P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:35.791566 28384 tablet_service.cc:1511] Processing CreateTablet for tablet 6903cb63c5b04408b7eaecf0c0231c42 (DEFAULT_TABLE table=default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.792968 28797 tablet_bootstrap.cc:492] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:35.793113 28797 ts_tablet_manager.cc:1403] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.061s user 0.001s sys 0.001s
I20260618 13:59:35.793834 28797 raft_consensus.cc:359] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.794029 28797 raft_consensus.cc:385] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.794106 28797 raft_consensus.cc:740] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.794353 28797 consensus_queue.cc:260] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.795770 28797 ts_tablet_manager.cc:1434] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:35.796069 28797 tablet_bootstrap.cc:492] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:35.797701 28797 tablet_bootstrap.cc:654] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:35.801167 28518 tablet_service.cc:1511] Processing CreateTablet for tablet 4a9a5f630654465fb416f53af1424e9b (DEFAULT_TABLE table=default.Table_642894fa43114390bed040b60b648d86 [id=b0963be8750947539a7cbadf5f0cb582]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.801751 28521 tablet_service.cc:1511] Processing CreateTablet for tablet 60df0bb1a20043049275fdc25084dc0d (DEFAULT_TABLE table=default.Table_0a233b0cc52a4af38b49f5b81dbf08d0 [id=2ae931fac0214533ac148c97afdc69d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.802289 28525 tablet_service.cc:1511] Processing CreateTablet for tablet d7c6c17ba06543219259749d7d38832d (DEFAULT_TABLE table=default.Table_11b11bd80db34f639f805447f8027a27 [id=be63cc607ee04972853447c9f771c131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.802721 28511 tablet_service.cc:1511] Processing CreateTablet for tablet 2128dbec33db453ba51070767a15b21e (DEFAULT_TABLE table=default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.803138 28527 tablet_service.cc:1511] Processing CreateTablet for tablet 6903cb63c5b04408b7eaecf0c0231c42 (DEFAULT_TABLE table=default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.803568 28523 tablet_service.cc:1511] Processing CreateTablet for tablet dbc88bf849a94cb9bd1c9c14aec53eb1 (DEFAULT_TABLE table=default.Table_bb918931281942b486860c5e4f5755f1 [id=74cfa445524642d9a804da39d46b6b32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.804613 28656 tablet_service.cc:1511] Processing CreateTablet for tablet 60df0bb1a20043049275fdc25084dc0d (DEFAULT_TABLE table=default.Table_0a233b0cc52a4af38b49f5b81dbf08d0 [id=2ae931fac0214533ac148c97afdc69d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.804760 28655 tablet_service.cc:1511] Processing CreateTablet for tablet 6903cb63c5b04408b7eaecf0c0231c42 (DEFAULT_TABLE table=default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.805003 28656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60df0bb1a20043049275fdc25084dc0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.805218 28663 tablet_service.cc:1511] Processing CreateTablet for tablet d7c6c17ba06543219259749d7d38832d (DEFAULT_TABLE table=default.Table_11b11bd80db34f639f805447f8027a27 [id=be63cc607ee04972853447c9f771c131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.805317 28654 tablet_service.cc:1511] Processing CreateTablet for tablet dbc88bf849a94cb9bd1c9c14aec53eb1 (DEFAULT_TABLE table=default.Table_bb918931281942b486860c5e4f5755f1 [id=74cfa445524642d9a804da39d46b6b32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.805577 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7c6c17ba06543219259749d7d38832d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.805824 28654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbc88bf849a94cb9bd1c9c14aec53eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.806108 29303 raft_consensus.cc:493] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.806241 29303 raft_consensus.cc:515] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.806633 28655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6903cb63c5b04408b7eaecf0c0231c42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.806854 29303 leader_election.cc:290] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.804786 28653 tablet_service.cc:1511] Processing CreateTablet for tablet 2128dbec33db453ba51070767a15b21e (DEFAULT_TABLE table=default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.807569 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "657e55fa53c447babfe91b1582806e6b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:35.807806 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2128dbec33db453ba51070767a15b21e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:35.808140 28618 leader_election.cc:343] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.810603 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "657e55fa53c447babfe91b1582806e6b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:35.810822 28407 raft_consensus.cc:2468] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:35.804929 28659 tablet_service.cc:1511] Processing CreateTablet for tablet 4a9a5f630654465fb416f53af1424e9b (DEFAULT_TABLE table=default.Table_642894fa43114390bed040b60b648d86 [id=b0963be8750947539a7cbadf5f0cb582]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.811251 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a9a5f630654465fb416f53af1424e9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.814952 28617 leader_election.cc:304] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.815892 28374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2128dbec33db453ba51070767a15b21e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.817663 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:35.817854 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:35.823486 30060 raft_consensus.cc:2804] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:35.823721 30060 raft_consensus.cc:493] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:35.823843 30060 raft_consensus.cc:3060] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.824378 28375 tablet_service.cc:1511] Processing CreateTablet for tablet dbc88bf849a94cb9bd1c9c14aec53eb1 (DEFAULT_TABLE table=default.Table_bb918931281942b486860c5e4f5755f1 [id=74cfa445524642d9a804da39d46b6b32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:35.824695 29914 raft_consensus.cc:493] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.824791 29914 raft_consensus.cc:515] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.825353 29914 leader_election.cc:290] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.826210 30060 raft_consensus.cc:515] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.826241 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa93f065b4d64e7b85e30f0690fae541" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:35.826401 29303 raft_consensus.cc:493] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.826594 29303 raft_consensus.cc:515] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:35.826890 28346 leader_election.cc:343] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.827418 30060 leader_election.cc:290] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.827914 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa93f065b4d64e7b85e30f0690fae541" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:35.827968 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "657e55fa53c447babfe91b1582806e6b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:35.828106 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa9f65d8a5c46fdb745ee17af946dce" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:35.828549 28618 leader_election.cc:343] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.827427 29303 leader_election.cc:290] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:35.828784 28618 leader_election.cc:343] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:35.829082 28346 leader_election.cc:343] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.829185 28346 leader_election.cc:304] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:35.829644 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa9f65d8a5c46fdb745ee17af946dce" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:35.829814 28411 raft_consensus.cc:2468] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:35.830675 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "657e55fa53c447babfe91b1582806e6b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:35.830808 28407 raft_consensus.cc:3060] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.835242 28617 leader_election.cc:304] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.835661 29303 raft_consensus.cc:2804] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:35.835752 29303 raft_consensus.cc:493] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:35.835844 29303 raft_consensus.cc:3060] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.836275 28797 tablet_bootstrap.cc:492] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:35.836431 28797 ts_tablet_manager.cc:1403] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.040s user 0.005s sys 0.000s
I20260618 13:59:35.837296 28797 raft_consensus.cc:359] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.837476 28797 raft_consensus.cc:385] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.837571 28797 raft_consensus.cc:740] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.837811 28797 consensus_queue.cc:260] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.838950 28797 ts_tablet_manager.cc:1434] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:35.839250 28797 tablet_bootstrap.cc:492] T f54910015987484e95b295352d859315 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:35.840898 28797 tablet_bootstrap.cc:654] T f54910015987484e95b295352d859315 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:35.841812 29914 raft_consensus.cc:2749] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:35.839191 28407 raft_consensus.cc:2468] T 657e55fa53c447babfe91b1582806e6b P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:35.843223 28617 leader_election.cc:304] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.843792 30060 raft_consensus.cc:2804] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:35.843925 30060 raft_consensus.cc:697] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:35.844182 30060 consensus_queue.cc:237] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.846799 29963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Bootstrap replayed 1/1 log segments. Stats: ops{read=509 overwritten=0 applied=500 ignored=0} inserts{seen=467 ignored=0} mutations{seen=588 ignored=0} orphaned_commits=0. Pending: 9 replicates
I20260618 13:59:35.848892 29963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Bootstrap complete.
I20260618 13:59:35.848986 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:35.849651 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 15180aefba0544329daee74308ef5010 (table default.Table_97b6bb39b3bf4612be391b9dd78d1646 [id=42a2a24590834abc9537669aaf223a7c]). Expected version 1 got 0
I20260618 13:59:35.838290 29303 raft_consensus.cc:515] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.851485 29303 leader_election.cc:290] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:35.851794 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa9f65d8a5c46fdb745ee17af946dce" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:35.852229 28797 tablet_bootstrap.cc:492] T f54910015987484e95b295352d859315 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
W20260618 13:59:35.852414 28618 leader_election.cc:343] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.852612 28797 ts_tablet_manager.cc:1403] T f54910015987484e95b295352d859315 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.001s
I20260618 13:59:35.853334 29566 ts_tablet_manager.cc:1929] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:35.853928 29566 log.cc:1199] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/dd2385e6752d49429422fc854539e9c6
I20260618 13:59:35.853369 28797 raft_consensus.cc:359] T f54910015987484e95b295352d859315 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.854212 28797 raft_consensus.cc:385] T f54910015987484e95b295352d859315 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:35.854302 28797 raft_consensus.cc:740] T f54910015987484e95b295352d859315 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.854652 29566 ts_tablet_manager.cc:1950] T dd2385e6752d49429422fc854539e9c6 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:35.856009 28797 consensus_queue.cc:260] T f54910015987484e95b295352d859315 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.856684 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 61) took 3809 ms. Trace:
I20260618 13:59:35.867424 29963 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 1 FOLLOWER]: Replica starting. Triggering 9 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:35.869758 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aa9f65d8a5c46fdb745ee17af946dce" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:35.870042 28411 raft_consensus.cc:3060] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.873515 29914 raft_consensus.cc:493] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.873633 29914 raft_consensus.cc:515] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.874224 29914 leader_election.cc:290] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.874976 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4c6537144d425f81fd1fafcd034b77" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:35.875828 28346 leader_election.cc:343] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.878666 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:35.879263 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:35.879712 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:35.880548 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_7f787ab851be496dae97cba8835db19b [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 1 got 0
I20260618 13:59:35.882784 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:35.887023 29963 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7637285570574a6dbcef1de6f84ecac0, State: Initialized, Role: FOLLOWER
I20260618 13:59:35.890998 29963 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 500, Last appended: 1.509, Last appended by leader: 509, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:35.865510 29303 raft_consensus.cc:493] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.871220 29714 leader_election.cc:290] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:35.875666 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4c6537144d425f81fd1fafcd034b77" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:35.895428 29303 raft_consensus.cc:515] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:35.892577 28617 consensus_peers.cc:597] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d502d67826ff4d7a8b0f515e5d8b6d77. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:35.896193 28346 leader_election.cc:343] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.896274 29303 leader_election.cc:290] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:35.896313 28346 leader_election.cc:304] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:35.897313 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab1ddf79b094ba2aeee9225c0b332c3" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:35.895407 29566 rpcz_store.cc:276] 0618 13:59:32.047423 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.047473 (+ 50us) service_pool.cc:224] Handling call
0618 13:59:35.419763 (+3372290us) ts_tablet_manager.cc:1151] Deleting tablet dd2385e6752d49429422fc854539e9c6
0618 13:59:35.419771 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:35.585241 (+165470us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:35.853239 (+267998us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:35.856647 (+ 3408us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3372115}
I20260618 13:59:35.897909 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet dd2385e6752d49429422fc854539e9c6 (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]) successfully deleted
W20260618 13:59:35.898068 28618 leader_election.cc:343] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.898818 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab1ddf79b094ba2aeee9225c0b332c3" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:35.899027 28409 raft_consensus.cc:2468] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:35.899070 28797 ts_tablet_manager.cc:1434] T f54910015987484e95b295352d859315 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.046s user 0.001s sys 0.001s
I20260618 13:59:35.899657 28797 tablet_bootstrap.cc:492] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:35.899839 28617 leader_election.cc:304] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.900292 30060 raft_consensus.cc:2804] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:35.900421 30060 raft_consensus.cc:493] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:35.900516 30060 raft_consensus.cc:3060] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.903299 30060 raft_consensus.cc:515] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.904686 29963 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 13:59:35.904831 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab1ddf79b094ba2aeee9225c0b332c3" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:35.905596 28618 leader_election.cc:343] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.907668 29914 raft_consensus.cc:2749] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:35.907665 30119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:35.908852 30119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 13:59:35.915982 29745 ts_tablet_manager.cc:1929] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:35.916435 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab1ddf79b094ba2aeee9225c0b332c3" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:35.916646 28409 raft_consensus.cc:3060] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.918159 29745 log.cc:1199] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/092a9aba9c6e48f5be9cf5d55ba93678
I20260618 13:59:35.918926 29566 tablet_replica.cc:333] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:35.919422 29566 raft_consensus.cc:2243] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:35.919893 28409 raft_consensus.cc:2468] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:35.920116 29566 raft_consensus.cc:2272] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:35.920590 28617 leader_election.cc:304] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.921036 29400 raft_consensus.cc:2804] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:35.921200 29400 raft_consensus.cc:697] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:35.921540 29400 consensus_queue.cc:237] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.922142 29914 raft_consensus.cc:493] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.922214 28411 raft_consensus.cc:2468] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:35.922283 29914 raft_consensus.cc:515] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.923059 29914 leader_election.cc:290] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.923121 28617 leader_election.cc:304] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.923588 29644 raft_consensus.cc:2804] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:35.923764 29644 raft_consensus.cc:697] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:35.924048 29644 consensus_queue.cc:237] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.924278 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2d54a901e9f4d5c89700979ce0618bf" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:35.924942 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2d54a901e9f4d5c89700979ce0618bf" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:35.925971 30060 leader_election.cc:290] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:35.925559 28346 leader_election.cc:343] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:35.926637 28346 leader_election.cc:343] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.926760 28346 leader_election.cc:304] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:35.927925 29914 raft_consensus.cc:2749] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:35.943821 29755 raft_consensus.cc:493] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.944036 29755 raft_consensus.cc:515] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.944825 29755 leader_election.cc:290] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:35.945410 30027 raft_consensus.cc:493] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.945580 30027 raft_consensus.cc:515] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.946311 30027 leader_election.cc:290] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:35.946815 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad045effb32948b5b8e9be5bb7996577" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:35.947105 28409 raft_consensus.cc:2468] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:35.947260 28484 leader_election.cc:343] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 5c75278db2234b11bf95f1cf5cc76edd
I20260618 13:59:35.947654 28617 leader_election.cc:304] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:35.947789 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad045effb32948b5b8e9be5bb7996577" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:35.948158 30060 raft_consensus.cc:2804] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:35.948316 30060 raft_consensus.cc:493] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260618 13:59:35.948388 28618 leader_election.cc:343] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:35.948509 30060 raft_consensus.cc:3060] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.952286 30060 raft_consensus.cc:515] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.953874 30060 leader_election.cc:290] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:35.954141 28409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad045effb32948b5b8e9be5bb7996577" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:35.954280 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad045effb32948b5b8e9be5bb7996577" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:35.954398 28409 raft_consensus.cc:3060] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:35.954958 28618 leader_election.cc:343] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:35.955310 28346 consensus_peers.cc:597] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:35.955427 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:35.956284 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:35.956944 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 72c04f55890648f0a8184f3783243f7c (table default.Table_773be314d7e741fcb4fb6eb7d954802e [id=0d83fc68255c458587390c58b12e00d7]). Expected version 2 got 0
I20260618 13:59:35.957398 28409 raft_consensus.cc:2468] T ad045effb32948b5b8e9be5bb7996577 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:35.958005 28617 leader_election.cc:304] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:35.964888 29714 raft_consensus.cc:2804] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:35.965090 29714 raft_consensus.cc:697] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:35.965377 29714 consensus_queue.cc:237] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.921541 29566 ts_tablet_manager.cc:1916] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:35.970119 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c75278db2234b11bf95f1cf5cc76edd" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:35.975181 29303 consensus_queue.cc:1048] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:35.971167 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6903cb63c5b04408b7eaecf0c0231c42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:35.975934 29914 raft_consensus.cc:493] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:35.977545 29914 raft_consensus.cc:515] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:35.978092 29303 consensus_queue.cc:1048] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:35.978260 29914 leader_election.cc:290] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:35.979952 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6dd422c05324c989e29b20ae2afabad" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:35.981200 28346 leader_election.cc:343] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: b6dd422c05324c989e29b20ae2afabad
I20260618 13:59:35.981329 28346 leader_election.cc:304] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:35.981544 29914 raft_consensus.cc:2749] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:35.982553 28375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbc88bf849a94cb9bd1c9c14aec53eb1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:35.985921 28618 consensus_peers.cc:597] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:35.988987 29759 raft_consensus.cc:493] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.989173 29759 raft_consensus.cc:515] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:35.989823 29759 leader_election.cc:290] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:35.991158 28484 leader_election.cc:343] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: dca54111fea047d9ae0fabe376d89382
W20260618 13:59:35.991487 28483 leader_election.cc:343] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: dca54111fea047d9ae0fabe376d89382
I20260618 13:59:35.991645 28483 leader_election.cc:304] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:35.992461 29759 raft_consensus.cc:2749] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:35.993067 29914 raft_consensus.cc:493] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:35.993273 29914 raft_consensus.cc:515] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:35.994815 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d8b3cfac0f648cd8b1c51102132c676" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:35.995632 28411 raft_consensus.cc:2393] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:35.996279 28483 leader_election.cc:304] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:35.996606 29759 raft_consensus.cc:3060] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:35.995666 29914 leader_election.cc:290] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:35.995968 28346 leader_election.cc:343] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.000468 29759 raft_consensus.cc:2749] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:36.000772 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d8b3cfac0f648cd8b1c51102132c676" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:36.001541 28346 leader_election.cc:343] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.001669 28346 leader_election.cc:304] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:36.001938 29914 raft_consensus.cc:2749] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:36.003810 30027 consensus_queue.cc:1048] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:36.012902 28618 consensus_peers.cc:597] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:36.031363 28618 consensus_peers.cc:597] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:36.034499 28618 consensus_peers.cc:597] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:36.044108 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:36.048827 28798 tablet_bootstrap.cc:654] T 7a9bd8577f4f4a9593a3ffeab582b02e P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.049554 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d43224c621a4774870d44f0237b514a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:36.053725 28511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2128dbec33db453ba51070767a15b21e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:36.058568 28521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60df0bb1a20043049275fdc25084dc0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:36.062582 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6903cb63c5b04408b7eaecf0c0231c42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:36.066553 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7c6c17ba06543219259749d7d38832d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:36.066817 29755 raft_consensus.cc:493] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.067039 29755 raft_consensus.cc:515] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:36.067772 28798 tablet_bootstrap.cc:492] T 7a9bd8577f4f4a9593a3ffeab582b02e P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:36.067800 29755 leader_election.cc:290] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:36.067970 28798 ts_tablet_manager.cc:1403] T 7a9bd8577f4f4a9593a3ffeab582b02e P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.277s user 0.026s sys 0.037s
I20260618 13:59:36.068768 28798 raft_consensus.cc:359] T 7a9bd8577f4f4a9593a3ffeab582b02e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.069103 28798 raft_consensus.cc:385] T 7a9bd8577f4f4a9593a3ffeab582b02e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.069305 28798 raft_consensus.cc:740] T 7a9bd8577f4f4a9593a3ffeab582b02e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
W20260618 13:59:36.069342 28483 leader_election.cc:343] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: ba0d05ea44e1473fb4b24f61eab096e4
I20260618 13:59:36.069841 28798 consensus_queue.cc:260] T 7a9bd8577f4f4a9593a3ffeab582b02e P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.070608 28798 ts_tablet_manager.cc:1434] T 7a9bd8577f4f4a9593a3ffeab582b02e P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:36.070960 28798 tablet_bootstrap.cc:492] T 04dac931e62a448d99633878eb70b92f P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:36.071326 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cbfebfca436461ab63dfe469ae14b4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:36.073134 28798 tablet_bootstrap.cc:654] T 04dac931e62a448d99633878eb70b92f P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.074602 28518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a9a5f630654465fb416f53af1424e9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:36.078557 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbc88bf849a94cb9bd1c9c14aec53eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:36.081636 28798 tablet_bootstrap.cc:492] T 04dac931e62a448d99633878eb70b92f P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:36.082154 28798 ts_tablet_manager.cc:1403] T 04dac931e62a448d99633878eb70b92f P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260618 13:59:36.083117 28798 raft_consensus.cc:359] T 04dac931e62a448d99633878eb70b92f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.084014 29745 ts_tablet_manager.cc:1950] T 092a9aba9c6e48f5be9cf5d55ba93678 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:36.084149 28798 raft_consensus.cc:385] T 04dac931e62a448d99633878eb70b92f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.084410 28798 raft_consensus.cc:740] T 04dac931e62a448d99633878eb70b92f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.084730 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 60) took 3178 ms. Trace:
I20260618 13:59:36.084937 28798 consensus_queue.cc:260] T 04dac931e62a448d99633878eb70b92f P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.085848 28798 ts_tablet_manager.cc:1434] T 04dac931e62a448d99633878eb70b92f P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260618 13:59:36.086611 28798 tablet_bootstrap.cc:492] T 92ddbdfdfff547588a03f453e413af8f P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:36.046389 28797 tablet_bootstrap.cc:654] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:36.047585 28618 consensus_peers.cc:597] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:36.089493 28798 tablet_bootstrap.cc:654] T 92ddbdfdfff547588a03f453e413af8f P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:36.091188 28618 consensus_peers.cc:597] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:36.091413 29714 raft_consensus.cc:493] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.091540 29714 raft_consensus.cc:515] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.092079 29714 leader_election.cc:290] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:36.090999 28484 leader_election.cc:343] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: ba0d05ea44e1473fb4b24f61eab096e4
I20260618 13:59:36.092352 28484 leader_election.cc:304] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:36.092643 29755 raft_consensus.cc:2749] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:36.093567 28618 consensus_peers.cc:597] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:36.093612 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18d8b609b96a4e81b57f69bff522eeea" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:36.094157 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a]). Expected version 1 got 0
W20260618 13:59:36.094215 28618 leader_election.cc:343] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.094509 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 39b06ae7444a48dbbec9e096fd7e4052 (table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15]). Expected version 2 got 0
I20260618 13:59:36.094732 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:36.094962 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:36.095232 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:36.095515 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:36.096196 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:36.096473 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:36.096720 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:36.099179 28798 tablet_bootstrap.cc:492] T 92ddbdfdfff547588a03f453e413af8f P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:36.099365 28798 ts_tablet_manager.cc:1403] T 92ddbdfdfff547588a03f453e413af8f P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.001s
I20260618 13:59:36.100258 28798 raft_consensus.cc:359] T 92ddbdfdfff547588a03f453e413af8f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.100455 28798 raft_consensus.cc:385] T 92ddbdfdfff547588a03f453e413af8f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.100538 28798 raft_consensus.cc:740] T 92ddbdfdfff547588a03f453e413af8f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.101891 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18d8b609b96a4e81b57f69bff522eeea" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:36.102062 28407 raft_consensus.cc:2468] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:36.102674 28617 leader_election.cc:304] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:36.102648 28798 consensus_queue.cc:260] T 92ddbdfdfff547588a03f453e413af8f P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.103035 30027 raft_consensus.cc:2804] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:36.103101 30027 raft_consensus.cc:493] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:36.103164 30027 raft_consensus.cc:3060] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.103662 28798 ts_tablet_manager.cc:1434] T 92ddbdfdfff547588a03f453e413af8f P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 13:59:36.105679 30060 consensus_queue.cc:1048] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260618 13:59:36.102146 29644 consensus_queue.cc:1048] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:36.110592 30027 raft_consensus.cc:515] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.111171 30027 leader_election.cc:290] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:36.084820 29745 rpcz_store.cc:276] 0618 13:59:32.906209 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.907955 (+ 1746us) service_pool.cc:224] Handling call
0618 13:59:35.589163 (+2681208us) ts_tablet_manager.cc:1151] Deleting tablet 092a9aba9c6e48f5be9cf5d55ba93678
0618 13:59:35.589174 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:35.871880 (+282706us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:35.915888 (+ 44008us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.084708 (+168820us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2681001}
I20260618 13:59:36.127238 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 092a9aba9c6e48f5be9cf5d55ba93678 (table default.Table_0a13b90182654346acbbc8f47a56436d [id=c31d65adb55f458c9ced14d598464f89]) successfully deleted
I20260618 13:59:36.127300 29745 tablet_replica.cc:333] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:36.127508 29745 raft_consensus.cc:2243] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:36.127696 29745 raft_consensus.cc:2272] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.128261 30027 raft_consensus.cc:493] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.128436 30027 raft_consensus.cc:515] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.129210 30027 leader_election.cc:290] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:36.129912 29745 ts_tablet_manager.cc:1916] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.136363 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18d8b609b96a4e81b57f69bff522eeea" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:36.136667 29894 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 1 FOLLOWER]: Advancing to term 2
I20260618 13:59:36.138916 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316110bd2d024c7599fa2933d87e559e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:36.140144 28618 leader_election.cc:343] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:36.141022 28618 consensus_peers.cc:597] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:36.141744 29894 pending_rounds.cc:85] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Aborting all ops after (but not including) 500
W20260618 13:59:36.141834 28618 leader_election.cc:343] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.154719 30125 raft_consensus.cc:493] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260618 13:59:36.154851 28618 consensus_peers.cc:597] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:36.154901 30125 raft_consensus.cc:515] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.169006 30125 leader_election.cc:290] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:36.169854 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340f6bbf18eb412ca3afec237ff7752d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:36.170195 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340f6bbf18eb412ca3afec237ff7752d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:36.170887 28346 leader_election.cc:343] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:36.171047 28346 leader_election.cc:343] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.171105 28346 leader_election.cc:304] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:36.171492 30125 raft_consensus.cc:2749] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:36.182052 28799 tablet_bootstrap.cc:492] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:36.182766 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18d8b609b96a4e81b57f69bff522eeea" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:36.182920 28411 raft_consensus.cc:3060] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.185350 28411 raft_consensus.cc:2468] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:36.185964 28548 raft_consensus.cc:3060] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.188479 28799 ts_tablet_manager.cc:1403] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.476s user 0.060s sys 0.113s
I20260618 13:59:36.197997 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316110bd2d024c7599fa2933d87e559e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:36.198199 28411 raft_consensus.cc:2468] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:36.199081 28457 heartbeater.cc:344] Connected to a master server at 127.27.100.60:42083
I20260618 13:59:36.199079 28617 leader_election.cc:304] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:36.199385 30125 raft_consensus.cc:493] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.199522 30125 raft_consensus.cc:515] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.199729 28617 leader_election.cc:304] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:36.200075 30027 raft_consensus.cc:2804] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:36.200117 30125 leader_election.cc:290] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:36.200201 30027 raft_consensus.cc:697] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:36.200425 30027 consensus_queue.cc:237] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.200990 29303 raft_consensus.cc:2804] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:36.200987 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a783590159f4a80964fbbcb4a24eef9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:36.201062 29303 raft_consensus.cc:493] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:36.201116 29303 raft_consensus.cc:3060] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.202065 29745 ts_tablet_manager.cc:1929] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.202148 29745 log.cc:1199] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/bce7bf74b6d24d0bb43808b8a62cf1b7
I20260618 13:59:36.202171 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a783590159f4a80964fbbcb4a24eef9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:36.202524 29745 ts_tablet_manager.cc:1950] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:36.202792 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 61) took 3296 ms. Trace:
I20260618 13:59:36.202864 29745 rpcz_store.cc:276] 0618 13:59:32.906309 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.908160 (+ 1851us) service_pool.cc:224] Handling call
0618 13:59:36.127221 (+3219061us) ts_tablet_manager.cc:1151] Deleting tablet bce7bf74b6d24d0bb43808b8a62cf1b7
0618 13:59:36.127235 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.197623 (+ 70388us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.202016 (+ 4393us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.202778 (+ 762us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3218879}
W20260618 13:59:36.202965 28346 leader_election.cc:343] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.203135 29745 tablet_replica.cc:333] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
W20260618 13:59:36.203163 28346 leader_election.cc:343] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.203249 28346 leader_election.cc:304] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:36.203330 29745 raft_consensus.cc:2243] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:36.203465 29745 raft_consensus.cc:2272] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.203778 29914 consensus_queue.cc:1048] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:36.203966 29745 ts_tablet_manager.cc:1916] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.204137 29914 raft_consensus.cc:2749] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:36.204109 29303 raft_consensus.cc:515] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.204974 29303 leader_election.cc:290] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:36.205513 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316110bd2d024c7599fa2933d87e559e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:36.206037 28618 leader_election.cc:343] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.206291 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316110bd2d024c7599fa2933d87e559e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:36.207818 28411 raft_consensus.cc:3060] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.207178 28799 raft_consensus.cc:359] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.210474 28799 raft_consensus.cc:385] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.210287 30125 consensus_queue.cc:1048] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:36.210812 28799 raft_consensus.cc:740] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.210305 28411 raft_consensus.cc:2468] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:36.211614 28617 leader_election.cc:304] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:36.212044 29714 raft_consensus.cc:2804] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:36.212162 29714 raft_consensus.cc:697] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:36.212407 29714 consensus_queue.cc:237] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.210343 29745 ts_tablet_manager.cc:1929] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.215263 29745 log.cc:1199] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/7d4144b1b5ea4e968672616bc22bead8
I20260618 13:59:36.216872 29745 ts_tablet_manager.cc:1950] T 7d4144b1b5ea4e968672616bc22bead8 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:36.211048 28799 consensus_queue.cc:260] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.217432 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 51) took 3312 ms. Trace:
I20260618 13:59:36.218055 28799 ts_tablet_manager.cc:1434] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.029s user 0.001s sys 0.001s
I20260618 13:59:36.218658 28799 tablet_bootstrap.cc:492] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:36.220554 28799 tablet_bootstrap.cc:654] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.224085 29745 rpcz_store.cc:276] 0618 13:59:32.905392 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.911081 (+ 5689us) service_pool.cc:224] Handling call
0618 13:59:36.203096 (+3292015us) ts_tablet_manager.cc:1151] Deleting tablet 7d4144b1b5ea4e968672616bc22bead8
0618 13:59:36.203104 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.207718 (+ 4614us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.210297 (+ 2579us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.217413 (+ 7116us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3291848}
I20260618 13:59:36.225275 30125 raft_consensus.cc:493] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.226300 29745 tablet_replica.cc:333] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:36.226300 30125 raft_consensus.cc:515] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.227689 29745 raft_consensus.cc:2243] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:36.229076 29745 raft_consensus.cc:2272] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.229461 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc295a8e81b64859915a6335f8fb8d8e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:36.230026 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc295a8e81b64859915a6335f8fb8d8e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:36.230676 28346 leader_election.cc:343] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:36.231016 28346 leader_election.cc:343] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.231169 28346 leader_election.cc:304] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:36.231511 29914 raft_consensus.cc:2749] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:36.228489 30125 leader_election.cc:290] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:36.232055 28799 tablet_bootstrap.cc:492] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:36.232200 28799 ts_tablet_manager.cc:1403] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.001s
I20260618 13:59:36.233536 28797 tablet_bootstrap.cc:492] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:36.234707 28799 raft_consensus.cc:359] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.234866 28799 raft_consensus.cc:385] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.234944 28799 raft_consensus.cc:740] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.236338 28797 ts_tablet_manager.cc:1403] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.337s user 0.002s sys 0.002s
I20260618 13:59:36.236640 28799 consensus_queue.cc:260] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.237299 28799 ts_tablet_manager.cc:1434] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260618 13:59:36.237620 28799 tablet_bootstrap.cc:492] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:36.239490 28799 tablet_bootstrap.cc:654] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.230988 29745 ts_tablet_manager.cc:1916] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.240517 29303 consensus_queue.cc:1048] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:36.243507 29749 raft_consensus.cc:493] T 7a9bd8577f4f4a9593a3ffeab582b02e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.243669 29749 raft_consensus.cc:515] T 7a9bd8577f4f4a9593a3ffeab582b02e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.244359 29749 leader_election.cc:290] T 7a9bd8577f4f4a9593a3ffeab582b02e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:36.245110 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9bd8577f4f4a9593a3ffeab582b02e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:36.246218 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9bd8577f4f4a9593a3ffeab582b02e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:36.246847 28484 leader_election.cc:304] T 7a9bd8577f4f4a9593a3ffeab582b02e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:36.247126 29749 raft_consensus.cc:2749] T 7a9bd8577f4f4a9593a3ffeab582b02e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:36.264889 28799 tablet_bootstrap.cc:492] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:36.265161 28799 ts_tablet_manager.cc:1403] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.028s user 0.004s sys 0.002s
W20260618 13:59:36.265707 28618 consensus_peers.cc:597] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:36.266732 28799 raft_consensus.cc:359] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.266963 28799 raft_consensus.cc:385] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.267046 28799 raft_consensus.cc:740] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.267310 28799 consensus_queue.cc:260] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.267824 28799 ts_tablet_manager.cc:1434] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:36.268110 28799 tablet_bootstrap.cc:492] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:36.271888 28797 raft_consensus.cc:359] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.272114 28797 raft_consensus.cc:385] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.272189 28797 raft_consensus.cc:740] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.273355 28799 tablet_bootstrap.cc:654] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:36.279102 28346 consensus_peers.cc:597] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:36.279359 29303 consensus_queue.cc:1048] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:36.280833 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet bce7bf74b6d24d0bb43808b8a62cf1b7 (table default.Table_3cdafadde7c146d38bdb87c358269c35 [id=214e8abbe81c4ff19009a4902bdef469]) successfully deleted
W20260618 13:59:36.281036 28988 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 77182.2us
I20260618 13:59:36.281298 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 7d4144b1b5ea4e968672616bc22bead8 (table default.Table_6ad6cf4da29143ff9ad3187cfcb95f6e [id=c2704c9f28bf4351a809b58bf4af056f]) successfully deleted
I20260618 13:59:36.282542 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c881dd3cb66344969310956ad6e83cc0 (table default.Table_0158779f563348c78b3505f98d43b839 [id=b1d85ea3aa7047d285ab7c86da8213fd]). Expected version 1 got 0
I20260618 13:59:36.283396 28798 tablet_bootstrap.cc:492] T c881dd3cb66344969310956ad6e83cc0 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:36.285434 28798 tablet_bootstrap.cc:654] T c881dd3cb66344969310956ad6e83cc0 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.285567 28797 consensus_queue.cc:260] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.293396 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:36.293708 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:36.293934 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:36.294140 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:36.294448 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:36.294950 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_c70d6c5b0dc94b33b4c5772edd2b9dca [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 2 got 0
I20260618 13:59:36.295239 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:36.295478 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:36.295882 29019 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:36.305349 30153 raft_consensus.cc:493] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.309828 29755 raft_consensus.cc:493] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:36.310045 29755 raft_consensus.cc:515] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.310863 29755 leader_election.cc:290] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:36.311861 28797 ts_tablet_manager.cc:1434] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.075s user 0.001s sys 0.001s
I20260618 13:59:36.312160 28797 tablet_bootstrap.cc:492] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:36.312171 28484 leader_election.cc:343] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: b6dd422c05324c989e29b20ae2afabad
W20260618 13:59:36.312948 28618 consensus_peers.cc:597] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b51d0851efb34eaa9e9e4be6b292ef62. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:36.312783 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6dd422c05324c989e29b20ae2afabad" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:36.313280 28407 raft_consensus.cc:2468] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:36.313709 28483 leader_election.cc:304] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2; no voters: d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:36.314154 29755 raft_consensus.cc:2804] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260618 13:59:36.314255 29755 raft_consensus.cc:493] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:36.314357 29755 raft_consensus.cc:3060] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Advancing to term 2
I20260618 13:59:36.314754 28798 tablet_bootstrap.cc:492] T c881dd3cb66344969310956ad6e83cc0 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:36.314766 28799 tablet_bootstrap.cc:492] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:36.314931 28797 tablet_bootstrap.cc:654] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.314934 28798 ts_tablet_manager.cc:1403] T c881dd3cb66344969310956ad6e83cc0 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.010s
I20260618 13:59:36.315083 28799 ts_tablet_manager.cc:1403] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.047s user 0.006s sys 0.000s
I20260618 13:59:36.315711 28798 raft_consensus.cc:359] T c881dd3cb66344969310956ad6e83cc0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.315871 28798 raft_consensus.cc:385] T c881dd3cb66344969310956ad6e83cc0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.315932 28798 raft_consensus.cc:740] T c881dd3cb66344969310956ad6e83cc0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.316083 28799 raft_consensus.cc:359] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.316176 28798 consensus_queue.cc:260] T c881dd3cb66344969310956ad6e83cc0 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.316287 28799 raft_consensus.cc:385] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.316349 28799 raft_consensus.cc:740] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
W20260618 13:59:36.316794 28983 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 438663f365e34fab87e34f434920cecf consensus state has no leader
I20260618 13:59:36.316579 28799 consensus_queue.cc:260] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.317103 28798 ts_tablet_manager.cc:1434] T c881dd3cb66344969310956ad6e83cc0 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:36.317198 29755 raft_consensus.cc:515] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.317353 28799 ts_tablet_manager.cc:1434] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:36.318351 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6dd422c05324c989e29b20ae2afabad" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:36.318557 28407 raft_consensus.cc:3060] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Advancing to term 2
I20260618 13:59:36.321691 28730 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:36.321867 28407 raft_consensus.cc:2468] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 2.
I20260618 13:59:36.322222 28483 leader_election.cc:304] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2; no voters:
I20260618 13:59:36.322532 29759 raft_consensus.cc:2804] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260618 13:59:36.322659 29759 raft_consensus.cc:697] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [term 2 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:36.322887 29759 consensus_queue.cc:237] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:36.323112 28484 leader_election.cc:343] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: b6dd422c05324c989e29b20ae2afabad
I20260618 13:59:36.324748 29755 leader_election.cc:290] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:36.326443 30152 raft_consensus.cc:493] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.326563 30152 raft_consensus.cc:515] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.327179 30152 leader_election.cc:290] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:36.328032 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab100ebbe61a4321b9e76c730e5d46ed" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:36.328511 28346 leader_election.cc:343] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.328810 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab100ebbe61a4321b9e76c730e5d46ed" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:36.329284 28346 leader_election.cc:343] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.329397 28346 leader_election.cc:304] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:36.329468 29303 consensus_queue.cc:1048] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:36.329802 30152 raft_consensus.cc:2749] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:36.330753 28798 tablet_bootstrap.cc:492] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:36.333253 28798 tablet_bootstrap.cc:654] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.345027 29755 raft_consensus.cc:493] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:36.345188 29755 raft_consensus.cc:515] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.345520 29749 raft_consensus.cc:493] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.345623 29749 raft_consensus.cc:515] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.346170 29749 leader_election.cc:290] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:36.347837 28483 leader_election.cc:343] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 316f470ea82f42fca48e4e96d49c9574
W20260618 13:59:36.348011 28484 leader_election.cc:343] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 316f470ea82f42fca48e4e96d49c9574
I20260618 13:59:36.348105 28484 leader_election.cc:304] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:36.348519 28483 leader_election.cc:343] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 94365c44b5e24acab5df60c67203794a
I20260618 13:59:36.348690 29749 raft_consensus.cc:2749] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260618 13:59:36.348664 28484 leader_election.cc:343] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 94365c44b5e24acab5df60c67203794a
I20260618 13:59:36.348834 28484 leader_election.cc:304] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:36.348862 29755 leader_election.cc:290] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:36.349027 29749 raft_consensus.cc:2749] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:36.349355 28799 tablet_bootstrap.cc:492] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:36.351034 28799 tablet_bootstrap.cc:654] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.351918 30153 raft_consensus.cc:515] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:36.354949 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b968a8d4014341ca9c5964263721ac79" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:36.355031 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b968a8d4014341ca9c5964263721ac79" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:36.356572 29566 ts_tablet_manager.cc:1929] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.356709 29566 log.cc:1199] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/d502d67826ff4d7a8b0f515e5d8b6d77
I20260618 13:59:36.357227 29566 ts_tablet_manager.cc:1950] T d502d67826ff4d7a8b0f515e5d8b6d77 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
W20260618 13:59:36.355545 28346 leader_election.cc:343] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.356451 28799 tablet_bootstrap.cc:492] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
W20260618 13:59:36.358479 28346 leader_election.cc:343] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.358493 28799 ts_tablet_manager.cc:1403] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260618 13:59:36.358608 28346 leader_election.cc:304] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:36.358848 30152 raft_consensus.cc:2749] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:36.352591 30153 leader_election.cc:290] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:36.359483 28799 raft_consensus.cc:359] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.359778 28799 raft_consensus.cc:385] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.359871 28799 raft_consensus.cc:740] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.360123 28799 consensus_queue.cc:260] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.360844 28799 ts_tablet_manager.cc:1434] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:36.361212 28799 tablet_bootstrap.cc:492] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:36.363451 28799 tablet_bootstrap.cc:654] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.364650 29894 pending_rounds.cc:107] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Aborting uncommitted WRITE_OP operation due to leader change: 1.501
I20260618 13:59:36.365577 29894 pending_rounds.cc:107] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Aborting uncommitted WRITE_OP operation due to leader change: 1.502
I20260618 13:59:36.365899 29894 pending_rounds.cc:107] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Aborting uncommitted WRITE_OP operation due to leader change: 1.503
I20260618 13:59:36.373119 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 62) took 4324 ms. Trace:
I20260618 13:59:36.377295 29566 rpcz_store.cc:276] 0618 13:59:32.048198 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.048269 (+ 71us) service_pool.cc:224] Handling call
0618 13:59:35.918749 (+3870480us) ts_tablet_manager.cc:1151] Deleting tablet d502d67826ff4d7a8b0f515e5d8b6d77
0618 13:59:35.918762 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.032161 (+113399us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.356488 (+324327us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.373093 (+ 16605us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3870299}
I20260618 13:59:36.377980 29566 tablet_replica.cc:333] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:36.378101 28799 tablet_bootstrap.cc:492] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:36.378135 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet d502d67826ff4d7a8b0f515e5d8b6d77 (table default.Table_b5800aca80594ec1bd3a86ec47dbfc64 [id=2710734517a24ad6baf0d47bf23e9bdf]) successfully deleted
I20260618 13:59:36.378242 28799 ts_tablet_manager.cc:1403] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.005s
I20260618 13:59:36.378877 29566 raft_consensus.cc:2243] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:36.379232 29566 raft_consensus.cc:2272] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.379455 28799 raft_consensus.cc:359] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.379668 29745 ts_tablet_manager.cc:1929] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.379701 28799 raft_consensus.cc:385] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.379967 28799 raft_consensus.cc:740] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.380187 28799 consensus_queue.cc:260] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.380807 28799 ts_tablet_manager.cc:1434] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260618 13:59:36.380954 29566 ts_tablet_manager.cc:1916] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.381068 28799 tablet_bootstrap.cc:492] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:36.382241 29714 consensus_queue.cc:1048] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:36.382841 28799 tablet_bootstrap.cc:654] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:36.403155 28618 consensus_peers.cc:597] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:36.403712 29746 ts_tablet_manager.cc:1929] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.403971 29746 log.cc:1199] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/4d6b01b897134d5bbd6a7d93b82ddb5e
I20260618 13:59:36.404480 29746 ts_tablet_manager.cc:1950] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
W20260618 13:59:36.404464 28618 consensus_peers.cc:597] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:36.405021 28797 tablet_bootstrap.cc:492] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:36.405206 28797 ts_tablet_manager.cc:1403] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.093s user 0.003s sys 0.000s
I20260618 13:59:36.406167 28797 raft_consensus.cc:359] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.406515 28797 raft_consensus.cc:385] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.406652 28797 raft_consensus.cc:740] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.407058 28797 consensus_queue.cc:260] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.409123 28797 ts_tablet_manager.cc:1434] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:36.409586 28797 tablet_bootstrap.cc:492] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:36.411365 29894 pending_rounds.cc:107] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Aborting uncommitted WRITE_OP operation due to leader change: 1.504
I20260618 13:59:36.412287 29894 pending_rounds.cc:107] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Aborting uncommitted WRITE_OP operation due to leader change: 1.505
I20260618 13:59:36.412631 29894 pending_rounds.cc:107] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Aborting uncommitted WRITE_OP operation due to leader change: 1.506
I20260618 13:59:36.413079 29894 pending_rounds.cc:107] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Aborting uncommitted WRITE_OP operation due to leader change: 1.507
I20260618 13:59:36.413561 29894 pending_rounds.cc:107] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Aborting uncommitted WRITE_OP operation due to leader change: 1.508
I20260618 13:59:36.414098 29894 pending_rounds.cc:107] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: Aborting uncommitted WRITE_OP operation due to leader change: 1.509
I20260618 13:59:36.419430 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 81) took 1861 ms. Trace:
I20260618 13:59:36.419582 29746 rpcz_store.cc:276] 0618 13:59:34.557881 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.557933 (+ 52us) service_pool.cc:224] Handling call
0618 13:59:35.656041 (+1098108us) ts_tablet_manager.cc:1151] Deleting tablet 4d6b01b897134d5bbd6a7d93b82ddb5e
0618 13:59:35.656074 (+ 33us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.173195 (+517121us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.368763 (+195568us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.419409 (+ 50646us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1097926}
I20260618 13:59:36.419983 30059 raft_consensus.cc:493] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.420127 30059 raft_consensus.cc:515] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:36.420210 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 4d6b01b897134d5bbd6a7d93b82ddb5e (table default.Table_45297e5c8f754fbbb3ab4dac6de1e136 [id=8c16c31ead484af3a1f3ee482b4cc233]) successfully deleted
I20260618 13:59:36.420683 29746 tablet_replica.cc:333] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:36.420742 30059 leader_election.cc:290] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:36.420902 29746 raft_consensus.cc:2243] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:36.421039 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bc87770baa34842a25ed97a19354446" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:36.421389 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bc87770baa34842a25ed97a19354446" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:36.421216 28407 raft_consensus.cc:2468] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:36.421921 28618 leader_election.cc:343] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.422039 28618 leader_election.cc:304] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:36.422231 30060 raft_consensus.cc:2804] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:36.422297 30060 raft_consensus.cc:493] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:36.422392 30060 raft_consensus.cc:3060] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.422528 29746 raft_consensus.cc:2272] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.423144 29746 ts_tablet_manager.cc:1916] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.430361 29566 ts_tablet_manager.cc:1929] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.431366 29566 log.cc:1199] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/751cd9bd961b4f56a588ed02a05f0212
I20260618 13:59:36.431761 29566 ts_tablet_manager.cc:1950] T 751cd9bd961b4f56a588ed02a05f0212 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:36.439004 28799 tablet_bootstrap.cc:492] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:36.439150 28799 ts_tablet_manager.cc:1403] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.058s user 0.001s sys 0.012s
I20260618 13:59:36.439973 28799 raft_consensus.cc:359] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.440152 28799 raft_consensus.cc:385] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.440213 28799 raft_consensus.cc:740] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.440403 28799 consensus_queue.cc:260] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.440809 28799 ts_tablet_manager.cc:1434] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:36.431937 30059 consensus_queue.cc:1048] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:36.441066 28799 tablet_bootstrap.cc:492] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:36.440965 30060 raft_consensus.cc:515] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:36.439707 29644 consensus_queue.cc:1048] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:36.442830 28799 tablet_bootstrap.cc:654] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.445600 28798 tablet_bootstrap.cc:492] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:36.445777 28798 ts_tablet_manager.cc:1403] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.115s user 0.013s sys 0.016s
I20260618 13:59:36.446570 28798 raft_consensus.cc:359] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.447356 28798 raft_consensus.cc:385] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.447471 28798 raft_consensus.cc:740] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.447753 28798 consensus_queue.cc:260] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.448305 28798 ts_tablet_manager.cc:1434] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:36.448303 29644 consensus_queue.cc:1048] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:36.451349 28798 tablet_bootstrap.cc:492] T be00bfa9bc1a4b83b18bfad05ba1495e P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:36.451627 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 64) took 4393 ms. Trace:
I20260618 13:59:36.451766 29566 rpcz_store.cc:276] 0618 13:59:32.058328 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.058382 (+ 54us) service_pool.cc:224] Handling call
0618 13:59:36.377931 (+4319549us) ts_tablet_manager.cc:1151] Deleting tablet 751cd9bd961b4f56a588ed02a05f0212
0618 13:59:36.377940 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.417807 (+ 39867us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.430311 (+ 12504us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.451607 (+ 21296us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4319356}
I20260618 13:59:36.452296 28799 tablet_bootstrap.cc:492] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:36.452272 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 751cd9bd961b4f56a588ed02a05f0212 (table default.Table_74620afb05b24bf9912f3984efd632aa [id=ad88f16ed9e143cda1e892de63a18849]) successfully deleted
I20260618 13:59:36.452430 28799 ts_tablet_manager.cc:1403] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260618 13:59:36.453078 28799 raft_consensus.cc:359] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.453230 28799 raft_consensus.cc:385] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.453289 28799 raft_consensus.cc:740] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.455437 30152 raft_consensus.cc:493] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.455576 30152 raft_consensus.cc:515] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.456156 30152 leader_election.cc:290] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:36.456660 28799 consensus_queue.cc:260] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.457187 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54910015987484e95b295352d859315" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:36.457818 28346 leader_election.cc:343] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.457851 28799 ts_tablet_manager.cc:1434] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260618 13:59:36.458166 28799 tablet_bootstrap.cc:492] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:36.458267 28798 tablet_bootstrap.cc:654] T be00bfa9bc1a4b83b18bfad05ba1495e P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.458525 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54910015987484e95b295352d859315" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:36.458695 28680 raft_consensus.cc:2468] T f54910015987484e95b295352d859315 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:36.459124 28346 leader_election.cc:304] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:36.459594 30152 raft_consensus.cc:2804] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:36.459676 30152 raft_consensus.cc:493] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:36.459756 30152 raft_consensus.cc:3060] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.460516 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bc87770baa34842a25ed97a19354446" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:36.460587 30060 leader_election.cc:290] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:36.460649 28403 raft_consensus.cc:3060] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.462911 29566 tablet_replica.cc:333] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:36.463184 28403 raft_consensus.cc:2468] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:36.463155 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bc87770baa34842a25ed97a19354446" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:36.463498 29566 raft_consensus.cc:2243] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:36.463574 28617 leader_election.cc:304] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
W20260618 13:59:36.463855 28618 leader_election.cc:343] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.464155 29566 pending_rounds.cc:70] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257: Trying to abort 1 pending ops.
I20260618 13:59:36.464221 29566 pending_rounds.cc:77] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298216656587337728 op_type: NO_OP noop_request { }
I20260618 13:59:36.464331 29566 raft_consensus.cc:2889] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:36.464428 29566 raft_consensus.cc:2272] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.465018 30152 raft_consensus.cc:515] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.465308 29566 ts_tablet_manager.cc:1916] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.466542 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54910015987484e95b295352d859315" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:36.466737 28680 raft_consensus.cc:3060] T f54910015987484e95b295352d859315 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.471361 29303 raft_consensus.cc:2804] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:36.471422 28680 raft_consensus.cc:2468] T f54910015987484e95b295352d859315 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:36.471514 29303 raft_consensus.cc:697] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:36.471791 28346 leader_election.cc:304] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:36.471788 29303 consensus_queue.cc:237] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:36.472749 28799 tablet_bootstrap.cc:654] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.472951 29914 raft_consensus.cc:2804] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:36.473088 29914 raft_consensus.cc:697] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:36.473366 29914 consensus_queue.cc:237] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.480370 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54910015987484e95b295352d859315" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:36.480912 28346 leader_election.cc:343] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.481115 30152 leader_election.cc:290] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:36.481984 29400 consensus_queue.cc:1048] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:36.484711 30152 raft_consensus.cc:493] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:36.484906 30152 raft_consensus.cc:515] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.486747 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aef7c1f111634c98af046e73fa9ffdeb" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:36.487138 28540 raft_consensus.cc:2468] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:36.487839 28346 leader_election.cc:304] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2; no voters:
I20260618 13:59:36.488194 30152 leader_election.cc:290] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:36.488408 30152 raft_consensus.cc:2804] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260618 13:59:36.488545 30152 raft_consensus.cc:493] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:36.488704 30152 raft_consensus.cc:3060] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Advancing to term 2
W20260618 13:59:36.491196 28346 leader_election.cc:343] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: aef7c1f111634c98af046e73fa9ffdeb
I20260618 13:59:36.494195 30152 raft_consensus.cc:515] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.494722 29566 ts_tablet_manager.cc:1929] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.494872 29566 log.cc:1199] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/7aa9f65d8a5c46fdb745ee17af946dce
I20260618 13:59:36.495335 29566 ts_tablet_manager.cc:1950] T 7aa9f65d8a5c46fdb745ee17af946dce P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:36.495882 29759 raft_consensus.cc:493] T c881dd3cb66344969310956ad6e83cc0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.496016 29759 raft_consensus.cc:515] T c881dd3cb66344969310956ad6e83cc0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.496559 29759 leader_election.cc:290] T c881dd3cb66344969310956ad6e83cc0 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:36.498127 28618 consensus_peers.cc:597] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:36.499559 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c881dd3cb66344969310956ad6e83cc0" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:36.503482 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 65) took 4442 ms. Trace:
I20260618 13:59:36.504288 29566 rpcz_store.cc:276] 0618 13:59:32.060676 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.060730 (+ 54us) service_pool.cc:224] Handling call
0618 13:59:36.452272 (+4391542us) ts_tablet_manager.cc:1151] Deleting tablet 7aa9f65d8a5c46fdb745ee17af946dce
0618 13:59:36.462867 (+ 10595us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.468431 (+ 5564us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.494647 (+ 26216us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.503465 (+ 8818us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4391378}
I20260618 13:59:36.504998 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 7aa9f65d8a5c46fdb745ee17af946dce (table default.Table_c2afd6643b7b49b7992384135658ddfc [id=29178416a03447608c28e809919d9461]) successfully deleted
I20260618 13:59:36.505115 29566 tablet_replica.cc:333] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:36.505386 29566 raft_consensus.cc:2243] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:36.506704 29566 raft_consensus.cc:2272] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.507779 29566 ts_tablet_manager.cc:1916] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.500437 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c881dd3cb66344969310956ad6e83cc0" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:36.523412 28483 leader_election.cc:304] T c881dd3cb66344969310956ad6e83cc0 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:36.524024 29759 raft_consensus.cc:2749] T c881dd3cb66344969310956ad6e83cc0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:36.379858 29745 log.cc:1199] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/4d6b01b897134d5bbd6a7d93b82ddb5e
I20260618 13:59:36.524995 29745 ts_tablet_manager.cc:1950] T 4d6b01b897134d5bbd6a7d93b82ddb5e P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:36.525653 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 67) took 3590 ms. Trace:
I20260618 13:59:36.525835 29745 rpcz_store.cc:276] 0618 13:59:32.935666 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.935936 (+ 270us) service_pool.cc:224] Handling call
0618 13:59:36.226242 (+3290306us) ts_tablet_manager.cc:1151] Deleting tablet 4d6b01b897134d5bbd6a7d93b82ddb5e
0618 13:59:36.226252 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.354969 (+128717us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.379612 (+ 24643us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.525633 (+146021us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3290117}
I20260618 13:59:36.526284 29566 ts_tablet_manager.cc:1929] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.526491 29566 log.cc:1199] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/d1baf86497d442dbbbddbc3498bebbee
I20260618 13:59:36.527297 29566 ts_tablet_manager.cc:1950] T d1baf86497d442dbbbddbc3498bebbee P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:36.527661 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 66) took 4466 ms. Trace:
I20260618 13:59:36.527793 29566 rpcz_store.cc:276] 0618 13:59:32.061574 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.061625 (+ 51us) service_pool.cc:224] Handling call
0618 13:59:36.505051 (+4443426us) ts_tablet_manager.cc:1151] Deleting tablet d1baf86497d442dbbbddbc3498bebbee
0618 13:59:36.505065 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.515734 (+ 10669us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.526226 (+ 10492us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.527646 (+ 1420us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4443267}
I20260618 13:59:36.528432 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet d1baf86497d442dbbbddbc3498bebbee (table default.Table_dac9c04f74664cce9c065edc3e491bf6 [id=5fe80671a240424da5ef1d7958caba2a]) successfully deleted
I20260618 13:59:36.528787 29566 tablet_replica.cc:333] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:36.529014 29566 raft_consensus.cc:2243] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:36.529379 29566 raft_consensus.cc:2272] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:36.529469 28618 consensus_peers.cc:597] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:36.531733 28797 tablet_bootstrap.cc:654] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.535347 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 4d6b01b897134d5bbd6a7d93b82ddb5e (table default.Table_45297e5c8f754fbbb3ab4dac6de1e136 [id=8c16c31ead484af3a1f3ee482b4cc233]) successfully deleted
I20260618 13:59:36.535717 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aef7c1f111634c98af046e73fa9ffdeb" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:36.535921 28540 raft_consensus.cc:3060] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Advancing to term 2
I20260618 13:59:36.536995 29566 ts_tablet_manager.cc:1916] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:36.540458 28346 leader_election.cc:343] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: aef7c1f111634c98af046e73fa9ffdeb
I20260618 13:59:36.538403 28540 raft_consensus.cc:2468] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 2.
I20260618 13:59:36.541134 28346 leader_election.cc:304] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2; no voters: d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:36.540329 28596 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:36.534878 30152 leader_election.cc:290] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 2 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:36.542095 30152 raft_consensus.cc:2804] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 2 FOLLOWER]: Leader election won for term 2
I20260618 13:59:36.542294 30152 raft_consensus.cc:697] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 2 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:36.542558 30152 consensus_queue.cc:237] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.537638 28798 tablet_bootstrap.cc:492] T be00bfa9bc1a4b83b18bfad05ba1495e P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:36.546666 28798 ts_tablet_manager.cc:1403] T be00bfa9bc1a4b83b18bfad05ba1495e P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.095s user 0.005s sys 0.003s
I20260618 13:59:36.546911 29566 ts_tablet_manager.cc:1929] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.547578 28798 raft_consensus.cc:359] T be00bfa9bc1a4b83b18bfad05ba1495e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:36.547834 28798 raft_consensus.cc:385] T be00bfa9bc1a4b83b18bfad05ba1495e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.547940 28798 raft_consensus.cc:740] T be00bfa9bc1a4b83b18bfad05ba1495e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.548237 28798 consensus_queue.cc:260] T be00bfa9bc1a4b83b18bfad05ba1495e P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:36.548588 29566 log.cc:1199] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/f89c9954d48e407c96c294c26644e25c
I20260618 13:59:36.549100 29566 ts_tablet_manager.cc:1950] T f89c9954d48e407c96c294c26644e25c P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:36.549486 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 67) took 4487 ms. Trace:
I20260618 13:59:36.549611 29566 rpcz_store.cc:276] 0618 13:59:32.062390 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.062455 (+ 65us) service_pool.cc:224] Handling call
0618 13:59:36.528742 (+4466287us) ts_tablet_manager.cc:1151] Deleting tablet f89c9954d48e407c96c294c26644e25c
0618 13:59:36.528752 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.541669 (+ 12917us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.546853 (+ 5184us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.549470 (+ 2617us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4466145}
I20260618 13:59:36.550122 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet f89c9954d48e407c96c294c26644e25c (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]) successfully deleted
I20260618 13:59:36.550513 29566 tablet_replica.cc:333] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:36.551121 29566 raft_consensus.cc:2243] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:36.551522 29566 raft_consensus.cc:2272] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.552074 29566 ts_tablet_manager.cc:1916] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.554512 28798 ts_tablet_manager.cc:1434] T be00bfa9bc1a4b83b18bfad05ba1495e P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260618 13:59:36.554834 28798 tablet_bootstrap.cc:492] T 450c68da26404767b01c5e8da3f6286d P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:36.556419 28798 tablet_bootstrap.cc:654] T 450c68da26404767b01c5e8da3f6286d P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.565857 29566 ts_tablet_manager.cc:1929] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.568459 29566 log.cc:1199] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/36b8a46a41e543c58a946b6764618c7d
I20260618 13:59:36.569015 29566 ts_tablet_manager.cc:1950] T 36b8a46a41e543c58a946b6764618c7d P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:36.569729 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 68) took 4506 ms. Trace:
I20260618 13:59:36.570204 29566 rpcz_store.cc:276] 0618 13:59:32.063243 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.063291 (+ 48us) service_pool.cc:224] Handling call
0618 13:59:36.550470 (+4487179us) ts_tablet_manager.cc:1151] Deleting tablet 36b8a46a41e543c58a946b6764618c7d
0618 13:59:36.550480 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.561120 (+ 10640us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.565806 (+ 4686us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.569713 (+ 3907us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4487028}
I20260618 13:59:36.571005 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 36b8a46a41e543c58a946b6764618c7d (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]) successfully deleted
I20260618 13:59:36.571162 30152 raft_consensus.cc:493] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.571421 29566 tablet_replica.cc:333] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:36.571363 30152 raft_consensus.cc:515] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.571879 29566 raft_consensus.cc:2243] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:36.572299 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b2f34bc17b4a27a91d6cafa9a96c48" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:36.572706 29566 raft_consensus.cc:2272] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:36.572786 28346 leader_election.cc:343] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.573436 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b2f34bc17b4a27a91d6cafa9a96c48" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:36.573841 28346 leader_election.cc:343] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.573941 28346 leader_election.cc:304] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:36.574170 29566 ts_tablet_manager.cc:1916] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.574560 30151 raft_consensus.cc:2749] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:36.575707 30152 leader_election.cc:290] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:36.584764 30152 raft_consensus.cc:493] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.585043 30152 raft_consensus.cc:515] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.586104 29566 ts_tablet_manager.cc:1929] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.586931 29566 log.cc:1199] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/bce7bf74b6d24d0bb43808b8a62cf1b7
I20260618 13:59:36.587062 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e99a85e83c46afa367e59e5ddb7dc7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:36.587700 28346 leader_election.cc:343] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.586189 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e99a85e83c46afa367e59e5ddb7dc7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:36.587870 29566 ts_tablet_manager.cc:1950] T bce7bf74b6d24d0bb43808b8a62cf1b7 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
W20260618 13:59:36.588404 28346 leader_election.cc:343] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.588521 28346 leader_election.cc:304] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:36.588914 30152 leader_election.cc:290] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:36.589251 30152 raft_consensus.cc:2749] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:36.589756 30165 raft_consensus.cc:493] T be00bfa9bc1a4b83b18bfad05ba1495e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.589893 30165 raft_consensus.cc:515] T be00bfa9bc1a4b83b18bfad05ba1495e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:36.590476 30165 leader_election.cc:290] T be00bfa9bc1a4b83b18bfad05ba1495e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:36.592761 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be00bfa9bc1a4b83b18bfad05ba1495e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:36.593868 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 69) took 4529 ms. Trace:
I20260618 13:59:36.594105 29566 rpcz_store.cc:276] 0618 13:59:32.064371 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.064419 (+ 48us) service_pool.cc:224] Handling call
0618 13:59:36.571373 (+4506954us) ts_tablet_manager.cc:1151] Deleting tablet bce7bf74b6d24d0bb43808b8a62cf1b7
0618 13:59:36.571383 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.577343 (+ 5960us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.586050 (+ 8707us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.593849 (+ 7799us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12032,"tablet-delete.queue_time_us":4506804}
I20260618 13:59:36.594671 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet bce7bf74b6d24d0bb43808b8a62cf1b7 (table default.Table_3cdafadde7c146d38bdb87c358269c35 [id=214e8abbe81c4ff19009a4902bdef469]) successfully deleted
I20260618 13:59:36.595043 29566 tablet_replica.cc:333] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:36.595281 29566 raft_consensus.cc:2243] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:36.596210 29749 raft_consensus.cc:493] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.595768 29566 raft_consensus.cc:2272] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.596331 29749 raft_consensus.cc:515] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.596503 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:36.596876 29749 leader_election.cc:290] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:36.597373 29566 ts_tablet_manager.cc:1916] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.597429 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5a334d18ba4c06bb358fa86d8af80d" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:36.598675 29020 catalog_manager.cc:5696] T b95bdb0e36584d92b9c28de257553827 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:36.599570 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:36.599946 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:36.600170 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:36.600436 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:36.600644 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:36.600994 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:36.602597 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5a334d18ba4c06bb358fa86d8af80d" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:36.603205 28483 leader_election.cc:304] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:36.603736 30165 raft_consensus.cc:2749] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:36.606096 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be00bfa9bc1a4b83b18bfad05ba1495e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:36.606632 28483 leader_election.cc:304] T be00bfa9bc1a4b83b18bfad05ba1495e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:36.607088 30060 raft_consensus.cc:493] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 97859748af2c41879ba2a668dda999d2)
I20260618 13:59:36.607204 30060 raft_consensus.cc:515] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.607671 30060 leader_election.cc:290] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:36.609126 28458 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:36.609316 29759 raft_consensus.cc:2749] T be00bfa9bc1a4b83b18bfad05ba1495e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:36.610831 28799 tablet_bootstrap.cc:492] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:36.610997 28799 ts_tablet_manager.cc:1403] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.153s user 0.028s sys 0.015s
I20260618 13:59:36.611689 28540 raft_consensus.cc:3060] T 7a9bd8577f4f4a9593a3ffeab582b02e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.611917 28799 raft_consensus.cc:359] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.612056 28799 raft_consensus.cc:385] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.612121 28799 raft_consensus.cc:740] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.612347 28799 consensus_queue.cc:260] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.612763 28799 ts_tablet_manager.cc:1434] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260618 13:59:36.615576 28618 leader_election.cc:343] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Not found: Tablet not found: da4d4f03911f4ac099ecc2fa1d8af494
I20260618 13:59:36.618357 29566 ts_tablet_manager.cc:1929] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.618559 29566 log.cc:1199] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/b0285c75c632457383e81805ab036aaf
I20260618 13:59:36.618980 29566 ts_tablet_manager.cc:1950] T b0285c75c632457383e81805ab036aaf P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:36.619436 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 70) took 4554 ms. Trace:
I20260618 13:59:36.619555 29566 rpcz_store.cc:276] 0618 13:59:32.065276 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.065321 (+ 45us) service_pool.cc:224] Handling call
0618 13:59:36.594997 (+4529676us) ts_tablet_manager.cc:1151] Deleting tablet b0285c75c632457383e81805ab036aaf
0618 13:59:36.595007 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.613501 (+ 18494us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.618313 (+ 4812us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.619415 (+ 1102us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4529515}
I20260618 13:59:36.619922 29566 tablet_replica.cc:333] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:36.620082 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet b0285c75c632457383e81805ab036aaf (table default.Table_9cd59394f11d4937887809241a69bba3 [id=75fcc464e22646f7960501f112b3f887]) successfully deleted
I20260618 13:59:36.620152 29566 raft_consensus.cc:2243] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:36.620313 29566 raft_consensus.cc:2272] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.620961 29566 ts_tablet_manager.cc:1916] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:36.628865 28484 consensus_peers.cc:597] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 -> Peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Couldn't send request to peer d6a169b16ced46ca9cfbc631709bb257. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:36.628046 29566 ts_tablet_manager.cc:1929] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.629199 29566 log.cc:1199] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/8b59255b14df4158afe37cb8890e7415
I20260618 13:59:36.629665 29566 ts_tablet_manager.cc:1950] T 8b59255b14df4158afe37cb8890e7415 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:36.629992 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 71) took 4563 ms. Trace:
I20260618 13:59:36.630113 29566 rpcz_store.cc:276] 0618 13:59:32.066314 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.066360 (+ 46us) service_pool.cc:224] Handling call
0618 13:59:36.619879 (+4553519us) ts_tablet_manager.cc:1151] Deleting tablet 8b59255b14df4158afe37cb8890e7415
0618 13:59:36.619889 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.624887 (+ 4998us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.628003 (+ 3116us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.629978 (+ 1975us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4553304}
I20260618 13:59:36.630666 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 8b59255b14df4158afe37cb8890e7415 (table default.Table_74c0d9b4f3724e278144c5fc30d8c2d7 [id=3bcc877eddec4d00b3dc1760cc635b36]) successfully deleted
I20260618 13:59:36.631024 29566 tablet_replica.cc:333] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:36.631114 28798 tablet_bootstrap.cc:492] T 450c68da26404767b01c5e8da3f6286d P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:36.631233 29566 raft_consensus.cc:2243] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:36.631563 29566 raft_consensus.cc:2272] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.631857 28797 tablet_bootstrap.cc:492] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:36.632104 29566 ts_tablet_manager.cc:1916] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.638094 28797 ts_tablet_manager.cc:1403] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.229s user 0.002s sys 0.003s
I20260618 13:59:36.638530 28798 ts_tablet_manager.cc:1403] T 450c68da26404767b01c5e8da3f6286d P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.084s user 0.002s sys 0.002s
I20260618 13:59:36.639710 28798 raft_consensus.cc:359] T 450c68da26404767b01c5e8da3f6286d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.639948 28798 raft_consensus.cc:385] T 450c68da26404767b01c5e8da3f6286d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.640007 28798 raft_consensus.cc:740] T 450c68da26404767b01c5e8da3f6286d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.640242 28798 consensus_queue.cc:260] T 450c68da26404767b01c5e8da3f6286d P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.644044 29746 ts_tablet_manager.cc:1929] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.644178 29746 log.cc:1199] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/516421cd843b44769efa2b555697c065
I20260618 13:59:36.644472 29566 ts_tablet_manager.cc:1929] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.644552 29746 ts_tablet_manager.cc:1950] T 516421cd843b44769efa2b555697c065 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:36.644665 29566 log.cc:1199] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/e95c88e4b0a046a3ad69ee49af47c7c3
I20260618 13:59:36.645146 29566 ts_tablet_manager.cc:1950] T e95c88e4b0a046a3ad69ee49af47c7c3 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:36.645339 28797 raft_consensus.cc:359] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.645541 28797 raft_consensus.cc:385] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.645623 28797 raft_consensus.cc:740] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.645866 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 72) took 4578 ms. Trace:
I20260618 13:59:36.647269 30060 consensus_queue.cc:1048] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:36.650776 28617 leader_election.cc:343] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: da4d4f03911f4ac099ecc2fa1d8af494
I20260618 13:59:36.650900 28617 leader_election.cc:304] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:36.651295 29714 raft_consensus.cc:2749] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:36.653829 29745 tablet_replica.cc:333] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:36.654031 29745 raft_consensus.cc:2243] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:36.654170 29745 raft_consensus.cc:2272] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.651259 29566 rpcz_store.cc:276] 0618 13:59:32.067503 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.067581 (+ 78us) service_pool.cc:224] Handling call
0618 13:59:36.630981 (+4563400us) ts_tablet_manager.cc:1151] Deleting tablet e95c88e4b0a046a3ad69ee49af47c7c3
0618 13:59:36.630991 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.641358 (+ 10367us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.644431 (+ 3073us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.645850 (+ 1419us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4563196}
I20260618 13:59:36.654609 29566 tablet_replica.cc:333] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:36.654670 29745 ts_tablet_manager.cc:1916] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.654796 29566 raft_consensus.cc:2243] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:36.654753 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet e95c88e4b0a046a3ad69ee49af47c7c3 (table default.Table_68ab7758379b44628a68285f209716ef [id=61ce11eead544e74a31db940c3376f4b]) successfully deleted
I20260618 13:59:36.654924 29566 raft_consensus.cc:2272] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.655525 29566 ts_tablet_manager.cc:1916] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.645839 28797 consensus_queue.cc:260] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.656703 28540 raft_consensus.cc:3060] T 04dac931e62a448d99633878eb70b92f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.658658 28799 tablet_bootstrap.cc:492] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:36.659188 28797 ts_tablet_manager.cc:1434] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.021s user 0.001s sys 0.000s
I20260618 13:59:36.659446 28797 tablet_bootstrap.cc:492] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:36.659962 28799 tablet_bootstrap.cc:654] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:36.660989 28618 consensus_peers.cc:597] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:36.662727 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:36.663007 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:36.664320 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:36.665347 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_7f787ab851be496dae97cba8835db19b [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 1 got 0
I20260618 13:59:36.665436 29745 ts_tablet_manager.cc:1929] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.665556 29745 log.cc:1199] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/d1baf86497d442dbbbddbc3498bebbee
I20260618 13:59:36.665678 29714 raft_consensus.cc:493] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.665812 29714 raft_consensus.cc:515] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.665935 29745 ts_tablet_manager.cc:1950] T d1baf86497d442dbbbddbc3498bebbee P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:36.665870 29020 catalog_manager.cc:5696] T a10e83d26b704201b95e7c0c44a1a4f5 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:36.666216 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 66) took 3730 ms. Trace:
I20260618 13:59:36.666366 29714 leader_election.cc:290] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:36.666289 29745 rpcz_store.cc:276] 0618 13:59:32.935538 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.936152 (+ 614us) service_pool.cc:224] Handling call
0618 13:59:36.535105 (+3598953us) ts_tablet_manager.cc:1151] Deleting tablet d1baf86497d442dbbbddbc3498bebbee
0618 13:59:36.653765 (+118660us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.657437 (+ 3672us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.665391 (+ 7954us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.666201 (+ 810us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3592478}
I20260618 13:59:36.667018 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ead386562eb4fd787c7aee776c2194f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:36.667163 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ead386562eb4fd787c7aee776c2194f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:36.667323 28403 raft_consensus.cc:2468] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:36.667788 28618 leader_election.cc:343] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.667968 28617 leader_election.cc:304] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:36.668252 29714 raft_consensus.cc:2804] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:36.668323 29714 raft_consensus.cc:493] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:36.668402 29714 raft_consensus.cc:3060] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.669104 29745 tablet_replica.cc:333] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:36.669286 29745 raft_consensus.cc:2243] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:36.669430 29745 raft_consensus.cc:2272] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.669907 29745 ts_tablet_manager.cc:1916] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.670696 29714 raft_consensus.cc:515] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.670905 28798 ts_tablet_manager.cc:1434] T 450c68da26404767b01c5e8da3f6286d P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.032s user 0.001s sys 0.001s
I20260618 13:59:36.671190 28798 tablet_bootstrap.cc:492] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:36.671674 29714 leader_election.cc:290] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:36.672147 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ead386562eb4fd787c7aee776c2194f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:36.672287 28403 raft_consensus.cc:3060] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.672377 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ead386562eb4fd787c7aee776c2194f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:36.673019 28798 tablet_bootstrap.cc:654] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.675031 28403 raft_consensus.cc:2468] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
W20260618 13:59:36.675612 28618 leader_election.cc:343] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.675642 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 82) took 2117 ms. Trace:
I20260618 13:59:36.675721 29746 rpcz_store.cc:276] 0618 13:59:34.558472 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.558523 (+ 51us) service_pool.cc:224] Handling call
0618 13:59:36.420634 (+1862111us) ts_tablet_manager.cc:1151] Deleting tablet 516421cd843b44769efa2b555697c065
0618 13:59:36.420645 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.559501 (+138856us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.643993 (+ 84492us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.675625 (+ 31632us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30266,"tablet-delete.queue_time_us":1861939}
I20260618 13:59:36.675900 28617 leader_election.cc:304] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:36.676420 29714 raft_consensus.cc:2804] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:36.676685 29714 raft_consensus.cc:697] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:36.676942 29714 consensus_queue.cc:237] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.677420 29566 ts_tablet_manager.cc:1929] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.677520 29566 log.cc:1199] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/37a6764a5e024903848561c9935b1a43
I20260618 13:59:36.678098 29566 ts_tablet_manager.cc:1950] T 37a6764a5e024903848561c9935b1a43 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:36.679441 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 73) took 4610 ms. Trace:
I20260618 13:59:36.679589 29566 rpcz_store.cc:276] 0618 13:59:32.068677 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.068742 (+ 65us) service_pool.cc:224] Handling call
0618 13:59:36.654567 (+4585825us) ts_tablet_manager.cc:1151] Deleting tablet 37a6764a5e024903848561c9935b1a43
0618 13:59:36.654578 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.663649 (+ 9071us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.677375 (+ 13726us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.679421 (+ 2046us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4585634}
I20260618 13:59:36.680017 29566 tablet_replica.cc:333] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:36.680243 29566 raft_consensus.cc:2243] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:36.680579 29566 raft_consensus.cc:2272] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.681154 29566 ts_tablet_manager.cc:1916] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.683580 29746 tablet_replica.cc:333] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:36.683851 29746 raft_consensus.cc:2243] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:36.684047 29746 raft_consensus.cc:2272] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.666800 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet d1baf86497d442dbbbddbc3498bebbee (table default.Table_dac9c04f74664cce9c065edc3e491bf6 [id=5fe80671a240424da5ef1d7958caba2a]) successfully deleted
I20260618 13:59:36.676333 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 516421cd843b44769efa2b555697c065 (table default.Table_c5c65f038c8f4f2783a705ffcc320629 [id=b06c685b8107494392219d12540a3a6e]) successfully deleted
I20260618 13:59:36.688565 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 37a6764a5e024903848561c9935b1a43 (table default.Table_88951dd073b74c99aa4a0bd7dcf2e19d [id=51cf184bc7634141841001c082262441]) successfully deleted
I20260618 13:59:36.689492 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:36.690064 29746 ts_tablet_manager.cc:1916] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.694950 29566 ts_tablet_manager.cc:1929] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.699060 28798 tablet_bootstrap.cc:492] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:36.700970 29566 log.cc:1199] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/fddea8fba4da40d7a5780e6750caf5e9
I20260618 13:59:36.701193 28729 heartbeater.cc:344] Connected to a master server at 127.27.100.60:42083
I20260618 13:59:36.701437 29566 ts_tablet_manager.cc:1950] T fddea8fba4da40d7a5780e6750caf5e9 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:36.661023 28797 tablet_bootstrap.cc:654] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.702813 28798 ts_tablet_manager.cc:1403] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.002s
I20260618 13:59:36.702845 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 74) took 4633 ms. Trace:
I20260618 13:59:36.703920 28798 raft_consensus.cc:359] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:36.705037 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:36.711670 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:36.712379 29566 rpcz_store.cc:276] 0618 13:59:32.069837 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.069924 (+ 87us) service_pool.cc:224] Handling call
0618 13:59:36.679978 (+4610054us) ts_tablet_manager.cc:1151] Deleting tablet fddea8fba4da40d7a5780e6750caf5e9
0618 13:59:36.679988 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.688969 (+ 8981us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.694906 (+ 5937us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.702829 (+ 7923us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4609861}
I20260618 13:59:36.713203 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 75) took 4641 ms. Trace:
I20260618 13:59:36.713336 29566 rpcz_store.cc:276] 0618 13:59:32.071355 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.071428 (+ 73us) service_pool.cc:224] Handling call
0618 13:59:36.713115 (+4641687us) ts_tablet_manager.cc:1151] Deleting tablet 4a01dfb4d59549399c94640f16045555
0618 13:59:36.713125 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.713189 (+ 64us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4641493}
I20260618 13:59:36.713745 29566 tablet_replica.cc:333] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:36.713968 29566 raft_consensus.cc:2243] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:36.714206 29566 pending_rounds.cc:70] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257: Trying to abort 1 pending ops.
I20260618 13:59:36.714478 28798 raft_consensus.cc:385] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.714468 29566 pending_rounds.cc:77] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298216657719271424 op_type: NO_OP noop_request { }
I20260618 13:59:36.714572 29566 raft_consensus.cc:2889] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:36.714769 28798 raft_consensus.cc:740] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.715309 28798 consensus_queue.cc:260] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:36.715804 29566 raft_consensus.cc:2272] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.716639 29566 ts_tablet_manager.cc:1916] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.717150 28798 ts_tablet_manager.cc:1434] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.014s user 0.001s sys 0.001s
I20260618 13:59:36.712960 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet fddea8fba4da40d7a5780e6750caf5e9 (table default.Table_4f1453c90c9f4b49b95607b12ad0bb65 [id=af94ae08478c4a37b443b5f46e4dd1c1]) successfully deleted
I20260618 13:59:36.717617 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
W20260618 13:59:36.717739 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 4a01dfb4d59549399c94640f16045555 because the tablet was not found. No further retry: Not found: Tablet not found: 4a01dfb4d59549399c94640f16045555
I20260618 13:59:36.717800 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:36.718104 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 15180aefba0544329daee74308ef5010 (table default.Table_97b6bb39b3bf4612be391b9dd78d1646 [id=42a2a24590834abc9537669aaf223a7c]). Expected version 1 got 0
I20260618 13:59:36.718389 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:36.719967 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_c70d6c5b0dc94b33b4c5772edd2b9dca [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 2 got 0
W20260618 13:59:36.723089 28618 consensus_peers.cc:597] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:36.723352 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:36.723548 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:36.723740 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:36.724000 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:36.724560 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 72c04f55890648f0a8184f3783243f7c (table default.Table_773be314d7e741fcb4fb6eb7d954802e [id=0d83fc68255c458587390c58b12e00d7]). Expected version 2 got 0
I20260618 13:59:36.728106 28798 tablet_bootstrap.cc:492] T a10e83d26b704201b95e7c0c44a1a4f5 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:36.729815 28798 tablet_bootstrap.cc:654] T a10e83d26b704201b95e7c0c44a1a4f5 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.739171 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:36.739665 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 39b06ae7444a48dbbec9e096fd7e4052 (table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15]). Expected version 2 got 0
I20260618 13:59:36.739944 28799 tablet_bootstrap.cc:492] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:36.740089 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a]). Expected version 1 got 0
I20260618 13:59:36.740134 28799 ts_tablet_manager.cc:1403] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.082s user 0.003s sys 0.010s
I20260618 13:59:36.740238 29746 ts_tablet_manager.cc:1929] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:36.740368 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:36.740440 29746 log.cc:1199] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/2f5a334d18ba4c06bb358fa86d8af80d
I20260618 13:59:36.740803 29746 ts_tablet_manager.cc:1950] T 2f5a334d18ba4c06bb358fa86d8af80d P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:36.740779 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:36.740850 28799 raft_consensus.cc:359] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.741376 28799 raft_consensus.cc:385] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.741201 29020 catalog_manager.cc:5696] T 92ddbdfdfff547588a03f453e413af8f P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:36.741940 29020 catalog_manager.cc:5696] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:36.742512 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:36.742774 28797 tablet_bootstrap.cc:492] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:36.742913 28797 ts_tablet_manager.cc:1403] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.084s user 0.003s sys 0.000s
I20260618 13:59:36.742920 29020 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:36.743667 28797 raft_consensus.cc:359] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.743834 28797 raft_consensus.cc:385] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.743894 28797 raft_consensus.cc:740] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.744161 28797 consensus_queue.cc:260] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.744786 28797 ts_tablet_manager.cc:1434] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:36.745095 28797 tablet_bootstrap.cc:492] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:36.745754 28799 raft_consensus.cc:740] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.746057 28799 consensus_queue.cc:260] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.749253 28797 tablet_bootstrap.cc:654] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.749442 29566 ts_tablet_manager.cc:1929] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.750239 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 83) took 2191 ms. Trace:
I20260618 13:59:36.750370 29746 rpcz_store.cc:276] 0618 13:59:34.559024 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.559074 (+ 50us) service_pool.cc:224] Handling call
0618 13:59:36.676202 (+2117128us) ts_tablet_manager.cc:1151] Deleting tablet 2f5a334d18ba4c06bb358fa86d8af80d
0618 13:59:36.683535 (+ 7333us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.698702 (+ 15167us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.740155 (+ 41453us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.750220 (+ 10065us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2116959}
I20260618 13:59:36.750906 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 2f5a334d18ba4c06bb358fa86d8af80d (table default.Table_2cb0539405824600bb10d2712fb17d31 [id=f90028f13e704830a408df1907fdae49]) successfully deleted
I20260618 13:59:36.756771 29745 ts_tablet_manager.cc:1929] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.756933 29745 log.cc:1199] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/b51d0851efb34eaa9e9e4be6b292ef62
I20260618 13:59:36.757350 29745 ts_tablet_manager.cc:1950] T b51d0851efb34eaa9e9e4be6b292ef62 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:36.757730 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 68) took 3821 ms. Trace:
I20260618 13:59:36.757851 29745 rpcz_store.cc:276] 0618 13:59:32.935793 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.936175 (+ 382us) service_pool.cc:224] Handling call
0618 13:59:36.669062 (+3732887us) ts_tablet_manager.cc:1151] Deleting tablet b51d0851efb34eaa9e9e4be6b292ef62
0618 13:59:36.669072 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.700100 (+ 31028us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.756722 (+ 56622us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.757715 (+ 993us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3726332}
I20260618 13:59:36.758296 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet b51d0851efb34eaa9e9e4be6b292ef62 (table default.Table_7d317fa28e3142a19d15c8360151c357 [id=bba3ab8821ca48a5aa1104094d78892a]) successfully deleted
I20260618 13:59:36.758378 29745 tablet_replica.cc:333] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:36.766331 29746 tablet_replica.cc:333] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:36.766628 29746 raft_consensus.cc:2243] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:36.766863 29746 raft_consensus.cc:2272] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.767452 29746 ts_tablet_manager.cc:1916] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.773404 29746 ts_tablet_manager.cc:1929] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.774583 29745 raft_consensus.cc:2243] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:36.774796 29745 raft_consensus.cc:2272] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.775331 29745 ts_tablet_manager.cc:1916] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.751552 28799 ts_tablet_manager.cc:1434] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260618 13:59:36.783671 28799 tablet_bootstrap.cc:492] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:36.786150 28799 tablet_bootstrap.cc:654] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.789762 29714 raft_consensus.cc:493] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.790526 29714 raft_consensus.cc:515] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.791145 29714 leader_election.cc:290] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:36.791853 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab100ebbe61a4321b9e76c730e5d46ed" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:36.792004 29566 log.cc:1199] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/18d8b609b96a4e81b57f69bff522eeea
I20260618 13:59:36.792021 28404 raft_consensus.cc:2468] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:36.792389 29566 ts_tablet_manager.cc:1950] T 18d8b609b96a4e81b57f69bff522eeea P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:36.793073 28617 leader_election.cc:304] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:36.793259 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 76) took 4720 ms. Trace:
I20260618 13:59:36.793344 29566 rpcz_store.cc:276] 0618 13:59:32.072466 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.072539 (+ 73us) service_pool.cc:224] Handling call
0618 13:59:36.713703 (+4641164us) ts_tablet_manager.cc:1151] Deleting tablet 18d8b609b96a4e81b57f69bff522eeea
0618 13:59:36.713712 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.742399 (+ 28687us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.749396 (+ 6997us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.793239 (+ 43843us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4640970}
I20260618 13:59:36.793751 29566 tablet_replica.cc:333] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:36.793934 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 18d8b609b96a4e81b57f69bff522eeea (table default.Table_1b32156053d845a5b192d2ccf583cac6 [id=60305da883dd4da2ab74ac31d5462bf0]) successfully deleted
I20260618 13:59:36.794407 29566 raft_consensus.cc:2243] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:36.794734 29566 raft_consensus.cc:2272] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.795320 29566 ts_tablet_manager.cc:1916] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.795485 29714 raft_consensus.cc:2804] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:36.795560 29714 raft_consensus.cc:493] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:36.795614 29714 raft_consensus.cc:3060] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.798249 29714 raft_consensus.cc:515] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.805914 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab100ebbe61a4321b9e76c730e5d46ed" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:36.806079 28404 raft_consensus.cc:3060] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.808936 28404 raft_consensus.cc:2468] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:36.811327 28799 tablet_bootstrap.cc:492] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:36.811403 28617 leader_election.cc:304] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:36.811565 28799 ts_tablet_manager.cc:1403] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.000s
I20260618 13:59:36.811831 30059 raft_consensus.cc:2804] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:36.811971 30059 raft_consensus.cc:697] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:36.812249 30059 consensus_queue.cc:237] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.812431 28799 raft_consensus.cc:359] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.812614 28799 raft_consensus.cc:385] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.812671 28799 raft_consensus.cc:740] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.813613 28799 consensus_queue.cc:260] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.814345 28799 ts_tablet_manager.cc:1434] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260618 13:59:36.814672 28799 tablet_bootstrap.cc:492] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:36.814810 29714 leader_election.cc:290] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:36.816023 28799 tablet_bootstrap.cc:654] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.776013 28546 raft_consensus.cc:3060] T 450c68da26404767b01c5e8da3f6286d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.791836 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab100ebbe61a4321b9e76c730e5d46ed" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:36.800463 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab100ebbe61a4321b9e76c730e5d46ed" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:36.820536 28618 leader_election.cc:343] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:36.820900 28618 leader_election.cc:343] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.802466 28595 heartbeater.cc:344] Connected to a master server at 127.27.100.60:42083
I20260618 13:59:36.814550 28549 raft_consensus.cc:3060] T be00bfa9bc1a4b83b18bfad05ba1495e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.823654 28797 tablet_bootstrap.cc:492] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:36.823765 28797 ts_tablet_manager.cc:1403] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.079s user 0.009s sys 0.009s
I20260618 13:59:36.824520 28797 raft_consensus.cc:359] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.824676 28797 raft_consensus.cc:385] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.824719 28797 raft_consensus.cc:740] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.824935 28797 consensus_queue.cc:260] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.825979 28797 ts_tablet_manager.cc:1434] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:36.826241 28797 tablet_bootstrap.cc:492] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:36.826439 29746 log.cc:1199] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/d9c00003300246ada2a356e3e1cf684b
I20260618 13:59:36.816030 28798 tablet_bootstrap.cc:492] T a10e83d26b704201b95e7c0c44a1a4f5 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:36.826815 29746 ts_tablet_manager.cc:1950] T d9c00003300246ada2a356e3e1cf684b P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:36.826849 28798 ts_tablet_manager.cc:1403] T a10e83d26b704201b95e7c0c44a1a4f5 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.099s user 0.010s sys 0.002s
I20260618 13:59:36.827651 28798 raft_consensus.cc:359] T a10e83d26b704201b95e7c0c44a1a4f5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.827898 28797 tablet_bootstrap.cc:654] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.828173 28798 raft_consensus.cc:385] T a10e83d26b704201b95e7c0c44a1a4f5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.828389 28798 raft_consensus.cc:740] T a10e83d26b704201b95e7c0c44a1a4f5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.828953 28798 consensus_queue.cc:260] T a10e83d26b704201b95e7c0c44a1a4f5 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.829380 29759 raft_consensus.cc:493] T a10e83d26b704201b95e7c0c44a1a4f5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.829478 29759 raft_consensus.cc:515] T a10e83d26b704201b95e7c0c44a1a4f5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:36.830039 29759 leader_election.cc:290] T a10e83d26b704201b95e7c0c44a1a4f5 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:36.830907 28798 ts_tablet_manager.cc:1434] T a10e83d26b704201b95e7c0c44a1a4f5 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 13:59:36.831558 28798 tablet_bootstrap.cc:492] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:36.831604 28548 raft_consensus.cc:3060] T 92ddbdfdfff547588a03f453e413af8f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.830857 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a10e83d26b704201b95e7c0c44a1a4f5" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:36.831890 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a10e83d26b704201b95e7c0c44a1a4f5" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:36.832584 28484 leader_election.cc:304] T a10e83d26b704201b95e7c0c44a1a4f5 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:36.834585 30165 raft_consensus.cc:2749] T a10e83d26b704201b95e7c0c44a1a4f5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:36.834830 30151 raft_consensus.cc:493] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.834942 30151 raft_consensus.cc:515] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.835537 30151 leader_election.cc:290] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:36.836442 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af2bec820b842b4bf6b57482007bd6a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:36.836922 28346 leader_election.cc:343] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.836462 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af2bec820b842b4bf6b57482007bd6a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:36.837847 28346 leader_election.cc:343] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.837949 28346 leader_election.cc:304] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:36.838326 30151 raft_consensus.cc:2749] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:36.844648 28797 tablet_bootstrap.cc:492] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:36.844900 28797 ts_tablet_manager.cc:1403] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.000s
I20260618 13:59:36.845142 28529 tablet_service.cc:1511] Processing CreateTablet for tablet 00a2abfda3ef40eca1f30ff7a1ec050e (DEFAULT_TABLE table=default.Table_9b57c986604640a3b9a634d8f979787e [id=6ea2cf9a99bf404fac0f0e3af7236d2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:36.845577 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00a2abfda3ef40eca1f30ff7a1ec050e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:36.845837 28797 raft_consensus.cc:359] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.845999 28797 raft_consensus.cc:385] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.846050 28797 raft_consensus.cc:740] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.846263 28797 consensus_queue.cc:260] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.846678 30059 consensus_queue.cc:1048] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:36.846849 28797 ts_tablet_manager.cc:1434] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:36.847126 28797 tablet_bootstrap.cc:492] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:36.848593 28797 tablet_bootstrap.cc:654] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.854298 28798 tablet_bootstrap.cc:654] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.859665 28653 tablet_service.cc:1511] Processing CreateTablet for tablet 00a2abfda3ef40eca1f30ff7a1ec050e (DEFAULT_TABLE table=default.Table_9b57c986604640a3b9a634d8f979787e [id=6ea2cf9a99bf404fac0f0e3af7236d2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:36.860123 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00a2abfda3ef40eca1f30ff7a1ec050e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:36.863436 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 00a2abfda3ef40eca1f30ff7a1ec050e (DEFAULT_TABLE table=default.Table_9b57c986604640a3b9a634d8f979787e [id=6ea2cf9a99bf404fac0f0e3af7236d2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:36.863869 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00a2abfda3ef40eca1f30ff7a1ec050e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:36.864096 28657 tablet_service.cc:1511] Processing CreateTablet for tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 (DEFAULT_TABLE table=default.Table_83d021ed148243ffb453cb5e2484b7b4 [id=b3cae943ecdf410ca1cf711c5ddfe14a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:36.864437 28657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:36.867347 28375 tablet_service.cc:1511] Processing CreateTablet for tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 (DEFAULT_TABLE table=default.Table_83d021ed148243ffb453cb5e2484b7b4 [id=b3cae943ecdf410ca1cf711c5ddfe14a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:36.867668 28375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:36.870936 28529 tablet_service.cc:1511] Processing CreateTablet for tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 (DEFAULT_TABLE table=default.Table_83d021ed148243ffb453cb5e2484b7b4 [id=b3cae943ecdf410ca1cf711c5ddfe14a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:36.871363 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:36.872422 28797 tablet_bootstrap.cc:492] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:36.872551 28797 ts_tablet_manager.cc:1403] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.001s
I20260618 13:59:36.873771 28797 raft_consensus.cc:359] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.873953 28797 raft_consensus.cc:385] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.874049 28797 raft_consensus.cc:740] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.874274 28797 consensus_queue.cc:260] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.875304 28797 ts_tablet_manager.cc:1434] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260618 13:59:36.875573 28797 tablet_bootstrap.cc:492] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:36.828027 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 84) took 2268 ms. Trace:
I20260618 13:59:36.876812 29746 rpcz_store.cc:276] 0618 13:59:34.559600 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.559652 (+ 52us) service_pool.cc:224] Handling call
0618 13:59:36.751179 (+2191527us) ts_tablet_manager.cc:1151] Deleting tablet d9c00003300246ada2a356e3e1cf684b
0618 13:59:36.766284 (+ 15105us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.770745 (+ 4461us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.773362 (+ 2617us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.828011 (+ 54649us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2191340}
I20260618 13:59:36.877362 28797 tablet_bootstrap.cc:654] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.877403 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet d9c00003300246ada2a356e3e1cf684b (table default.Table_a9703f9d3b54453691c354131f824e22 [id=f298b441bd9b444b9edd832be0adb03a]) successfully deleted
I20260618 13:59:36.878607 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 85) took 2318 ms. Trace:
I20260618 13:59:36.879439 28523 tablet_service.cc:1511] Processing CreateTablet for tablet e44ad8cb18de42e795da5f229f49cdfe (DEFAULT_TABLE table=default.Table_795f17a0755242fba8eb1a55877bd470 [id=bc3f737180874cafa41b6903956e44f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:36.879756 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e44ad8cb18de42e795da5f229f49cdfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:36.880512 29745 ts_tablet_manager.cc:1929] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.880597 29745 log.cc:1199] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/aa8cf927bed9437086efd8e0f21ba26a
I20260618 13:59:36.880908 29745 ts_tablet_manager.cc:1950] T aa8cf927bed9437086efd8e0f21ba26a P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:36.887900 28653 tablet_service.cc:1511] Processing CreateTablet for tablet e44ad8cb18de42e795da5f229f49cdfe (DEFAULT_TABLE table=default.Table_795f17a0755242fba8eb1a55877bd470 [id=bc3f737180874cafa41b6903956e44f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:36.888378 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e44ad8cb18de42e795da5f229f49cdfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:36.890862 30186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a950ab7cd66b4cb0b29592aa692c2dd0. Latest consensus state: current_term: 2 leader_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:36.891108 30181 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a950ab7cd66b4cb0b29592aa692c2dd0)
I20260618 13:59:36.891219 30186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 13:59:36.891368 30181 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:36.895274 30181 leader_election.cc:290] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e4d4f4566be42dfbafb7c0e40fb17ff (127.27.100.62:43001), a950ab7cd66b4cb0b29592aa692c2dd0 (127.27.100.61:45967)
W20260618 13:59:36.897094 28617 consensus_peers.cc:597] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7d4144b1b5ea4e968672616bc22bead8. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:36.899277 29714 raft_consensus.cc:493] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.899468 29714 raft_consensus.cc:515] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.900163 29714 leader_election.cc:290] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:36.901636 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d8b3cfac0f648cd8b1c51102132c676" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:36.902853 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d8b3cfac0f648cd8b1c51102132c676" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:36.892493 29894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40964 (request call id 0) took 2817 ms. Trace:
W20260618 13:59:36.903636 28618 leader_election.cc:343] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.905099 30218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:36.905398 30218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 13:59:36.878742 29746 rpcz_store.cc:276] 0618 13:59:34.560165 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.560216 (+ 51us) service_pool.cc:224] Handling call
0618 13:59:36.878488 (+2318272us) ts_tablet_manager.cc:1151] Deleting tablet c4cfca9e76e641a7bc0b58b9ee240f36
0618 13:59:36.878498 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.878589 (+ 91us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2318101}
W20260618 13:59:36.906461 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet c4cfca9e76e641a7bc0b58b9ee240f36 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c4cfca9e76e641a7bc0b58b9ee240f36 already in progress: creating tablet
I20260618 13:59:36.906925 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 86) took 2346 ms. Trace:
I20260618 13:59:36.907048 29746 rpcz_store.cc:276] 0618 13:59:34.560717 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.560768 (+ 51us) service_pool.cc:224] Handling call
0618 13:59:36.906786 (+2346018us) ts_tablet_manager.cc:1151] Deleting tablet f3ba9186b3ff47d5a2b158fef619686e
0618 13:59:36.906799 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.906905 (+ 106us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2345842}
W20260618 13:59:36.907698 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet f3ba9186b3ff47d5a2b158fef619686e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f3ba9186b3ff47d5a2b158fef619686e already in progress: creating tablet
I20260618 13:59:36.908185 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 87) took 2346 ms. Trace:
I20260618 13:59:36.908293 29746 rpcz_store.cc:276] 0618 13:59:34.561273 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.561323 (+ 50us) service_pool.cc:224] Handling call
0618 13:59:36.908064 (+2346741us) ts_tablet_manager.cc:1151] Deleting tablet c3d523419ff244f085d6eac1a3231973
0618 13:59:36.908076 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.908166 (+ 90us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2346582}
W20260618 13:59:36.908742 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet c3d523419ff244f085d6eac1a3231973 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c3d523419ff244f085d6eac1a3231973 already in progress: creating tablet
I20260618 13:59:36.892158 30225 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260618 13:59:36.918080 28540 raft_consensus.cc:3060] T c881dd3cb66344969310956ad6e83cc0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:36.923583 28617 consensus_peers.cc:597] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b51d0851efb34eaa9e9e4be6b292ef62. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:36.926908 28097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7637285570574a6dbcef1de6f84ecac0" candidate_term: 3 candidate_status { last_received { term: 2 index: 584 } } ignore_live_leader: false dest_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" is_pre_election: true
I20260618 13:59:36.930583 30231 tablet.cc:1724] T c881dd3cb66344969310956ad6e83cc0 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:36.936311 28797 tablet_bootstrap.cc:492] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:36.936677 28797 ts_tablet_manager.cc:1403] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.061s user 0.001s sys 0.004s
I20260618 13:59:36.937863 28797 raft_consensus.cc:359] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.938089 28797 raft_consensus.cc:385] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.938174 28797 raft_consensus.cc:740] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.938445 28797 consensus_queue.cc:260] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.944849 30059 consensus_queue.cc:1048] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:36.945567 28797 ts_tablet_manager.cc:1434] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.009s user 0.000s sys 0.002s
I20260618 13:59:36.946331 28797 tablet_bootstrap.cc:492] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:36.949658 30151 raft_consensus.cc:493] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.954104 28403 raft_consensus.cc:2468] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:36.954310 28383 tablet_service.cc:1511] Processing CreateTablet for tablet e44ad8cb18de42e795da5f229f49cdfe (DEFAULT_TABLE table=default.Table_795f17a0755242fba8eb1a55877bd470 [id=bc3f737180874cafa41b6903956e44f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:36.954670 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e44ad8cb18de42e795da5f229f49cdfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:36.954795 29048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7637285570574a6dbcef1de6f84ecac0" candidate_term: 3 candidate_status { last_received { term: 2 index: 584 } } ignore_live_leader: false dest_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" is_pre_election: true
I20260618 13:59:36.958676 29806 leader_election.cc:304] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7637285570574a6dbcef1de6f84ecac0; no voters: 9e4d4f4566be42dfbafb7c0e40fb17ff, a950ab7cd66b4cb0b29592aa692c2dd0
I20260618 13:59:36.959179 30218 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260618 13:59:36.960686 30060 raft_consensus.cc:493] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:36.960745 28617 leader_election.cc:304] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:36.960868 30060 raft_consensus.cc:515] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.960964 29303 raft_consensus.cc:2804] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:36.961058 29303 raft_consensus.cc:493] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:36.961138 29303 raft_consensus.cc:3060] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.961475 30060 leader_election.cc:290] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:36.961884 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56650c9e20e24a28821cca4279bfde58" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:36.962116 28799 tablet_bootstrap.cc:492] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:36.962217 28799 ts_tablet_manager.cc:1403] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.148s user 0.003s sys 0.000s
I20260618 13:59:36.962906 28799 raft_consensus.cc:359] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.963053 28799 raft_consensus.cc:385] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:36.963100 28799 raft_consensus.cc:740] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:36.960667 28797 tablet_bootstrap.cc:654] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:36.963307 28799 consensus_queue.cc:260] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.963748 28799 ts_tablet_manager.cc:1434] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260618 13:59:36.964012 28799 tablet_bootstrap.cc:492] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:36.965456 29303 raft_consensus.cc:515] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.965500 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56650c9e20e24a28821cca4279bfde58" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:36.965814 30151 raft_consensus.cc:515] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.966337 30151 leader_election.cc:290] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:36.966516 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 64) took 4003 ms. Trace:
I20260618 13:59:36.966631 29745 rpcz_store.cc:276] 0618 13:59:32.935347 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.936200 (+ 853us) service_pool.cc:224] Handling call
0618 13:59:36.758334 (+3822134us) ts_tablet_manager.cc:1151] Deleting tablet aa8cf927bed9437086efd8e0f21ba26a
0618 13:59:36.758344 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.778192 (+ 19848us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.880470 (+102278us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.938651 (+ 58181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15871,"tablet-delete.queue_time_us":3815590}
I20260618 13:59:36.966786 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56650c9e20e24a28821cca4279bfde58" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:36.966938 28539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d8b3cfac0f648cd8b1c51102132c676" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:36.967430 29745 tablet_replica.cc:333] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:36.967629 29745 raft_consensus.cc:2243] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:36.967927 29745 raft_consensus.cc:2272] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.967767 28404 raft_consensus.cc:2468] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:36.968813 29745 ts_tablet_manager.cc:1916] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.970177 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d8b3cfac0f648cd8b1c51102132c676" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:36.970438 28404 raft_consensus.cc:3060] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:36.970860 28346 leader_election.cc:343] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.971100 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet aa8cf927bed9437086efd8e0f21ba26a (table default.Table_cf80fd5bbdde4cfb855c8fbe860a15f3 [id=9d63ad7327a44124af817b465a9ae0da]) successfully deleted
I20260618 13:59:36.974116 29745 ts_tablet_manager.cc:1929] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.974223 29745 log.cc:1199] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/e95c88e4b0a046a3ad69ee49af47c7c3
I20260618 13:59:36.974666 29746 tablet_replica.cc:333] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:36.975090 29746 raft_consensus.cc:2243] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:36.975291 29746 raft_consensus.cc:2272] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.975835 29746 ts_tablet_manager.cc:1916] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.981652 29870 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" instance_seqno: 1781791158100017) as {username='slave'} at 127.27.100.1:60185; Asking this server to re-register.
I20260618 13:59:36.982816 29746 ts_tablet_manager.cc:1929] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:36.982918 29746 log.cc:1199] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/fddea8fba4da40d7a5780e6750caf5e9
I20260618 13:59:36.984225 29853 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "97859748af2c41879ba2a668dda999d2" instance_seqno: 1781791158554469) as {username='slave'} at 127.27.100.2:47913; Asking this server to re-register.
I20260618 13:59:36.986644 29746 ts_tablet_manager.cc:1950] T fddea8fba4da40d7a5780e6750caf5e9 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:36.987691 28595 heartbeater.cc:461] Registering TS with master...
I20260618 13:59:36.988507 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 88) took 2426 ms. Trace:
I20260618 13:59:36.988937 29746 rpcz_store.cc:276] 0618 13:59:34.561826 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.561878 (+ 52us) service_pool.cc:224] Handling call
0618 13:59:36.974621 (+2412743us) ts_tablet_manager.cc:1151] Deleting tablet fddea8fba4da40d7a5780e6750caf5e9
0618 13:59:36.974629 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.980030 (+ 5401us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.982762 (+ 2732us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.988487 (+ 5725us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2412580}
I20260618 13:59:36.989238 28457 heartbeater.cc:461] Registering TS with master...
I20260618 13:59:36.989693 29746 tablet_replica.cc:333] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:36.989882 29746 raft_consensus.cc:2243] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:36.990192 29746 raft_consensus.cc:2272] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:36.990753 30059 consensus_queue.cc:1048] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260618 13:59:36.990924 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet fddea8fba4da40d7a5780e6750caf5e9 (table default.Table_4f1453c90c9f4b49b95607b12ad0bb65 [id=af94ae08478c4a37b443b5f46e4dd1c1]) successfully deleted
I20260618 13:59:36.991551 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56650c9e20e24a28821cca4279bfde58" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:36.992511 28680 raft_consensus.cc:2468] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:36.993046 28346 leader_election.cc:304] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:36.993429 30151 raft_consensus.cc:2804] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:36.993511 30151 raft_consensus.cc:493] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:36.993572 30151 raft_consensus.cc:3060] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:36.994524 29746 ts_tablet_manager.cc:1916] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:36.994791 29303 leader_election.cc:290] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:36.994920 28618 leader_election.cc:343] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.996078 30151 raft_consensus.cc:515] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:36.996714 30151 leader_election.cc:290] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:36.997318 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56650c9e20e24a28821cca4279bfde58" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:36.997671 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56650c9e20e24a28821cca4279bfde58" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:36.998158 28346 leader_election.cc:343] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:36.998867 28680 raft_consensus.cc:3060] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.000304 29853 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" instance_seqno: 1781791159034721) as {username='slave'} at 127.27.100.3:39823; Asking this server to re-register.
I20260618 13:59:37.001391 28680 raft_consensus.cc:2468] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:37.001700 29759 raft_consensus.cc:493] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.001878 28346 leader_election.cc:304] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.001832 29759 raft_consensus.cc:515] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.002390 29759 leader_election.cc:290] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.002578 30151 raft_consensus.cc:2804] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:37.002705 30151 raft_consensus.cc:697] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
W20260618 13:59:37.003093 28618 leader_election.cc:343] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.003058 30151 consensus_queue.cc:237] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.003700 28617 leader_election.cc:304] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
W20260618 13:59:37.003858 28483 leader_election.cc:343] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: fca76774f119435899188313a33dad98
I20260618 13:59:37.004108 29714 raft_consensus.cc:2764] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260618 13:59:37.006330 28729 heartbeater.cc:461] Registering TS with master...
W20260618 13:59:37.008008 28618 consensus_peers.cc:597] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:37.009804 29566 ts_tablet_manager.cc:1929] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.009997 29566 log.cc:1199] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/d667ea5f029f46d89fcb3be03069b97b
I20260618 13:59:37.010485 29566 ts_tablet_manager.cc:1950] T d667ea5f029f46d89fcb3be03069b97b P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:37.011649 28797 tablet_bootstrap.cc:492] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:37.011860 28797 ts_tablet_manager.cc:1403] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.066s user 0.001s sys 0.003s
I20260618 13:59:37.012645 28797 raft_consensus.cc:359] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.012912 28797 raft_consensus.cc:385] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260618 13:59:37.013928 28484 leader_election.cc:343] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: fca76774f119435899188313a33dad98
I20260618 13:59:37.014067 28484 leader_election.cc:304] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.014411 29759 raft_consensus.cc:2749] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:37.018978 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 77) took 4944 ms. Trace:
I20260618 13:59:37.019119 29566 rpcz_store.cc:276] 0618 13:59:32.074273 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.074367 (+ 94us) service_pool.cc:224] Handling call
0618 13:59:36.793703 (+4719336us) ts_tablet_manager.cc:1151] Deleting tablet d667ea5f029f46d89fcb3be03069b97b
0618 13:59:36.793714 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.833766 (+ 40052us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.975432 (+141666us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.018931 (+ 43499us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4719112}
I20260618 13:59:37.019677 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet d667ea5f029f46d89fcb3be03069b97b (table default.Table_f4a03a199e51400ca4c8a5d868ec2025 [id=0ce6f37b68d8435f8f860ba43647bc3b]) successfully deleted
I20260618 13:59:37.019745 29853 ts_manager.cc:194] Registered new tserver with Master: 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.020102 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 78) took 4944 ms. Trace:
I20260618 13:59:37.020243 29566 rpcz_store.cc:276] 0618 13:59:32.075461 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.075525 (+ 64us) service_pool.cc:224] Handling call
0618 13:59:37.020022 (+4944497us) ts_tablet_manager.cc:1151] Deleting tablet 69d5e75b4a8247b29d69d07fad5ca5db
0618 13:59:37.020031 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.020087 (+ 56us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4944037}
W20260618 13:59:37.020782 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 69d5e75b4a8247b29d69d07fad5ca5db because the tablet was not found. No further retry: Not found: Tablet not found: 69d5e75b4a8247b29d69d07fad5ca5db
I20260618 13:59:37.021102 29566 tablet_replica.cc:333] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:37.021351 29566 raft_consensus.cc:2243] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:37.021677 29566 raft_consensus.cc:2272] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.022244 29566 ts_tablet_manager.cc:1916] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.022346 29746 ts_tablet_manager.cc:1929] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.022480 29746 log.cc:1199] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/37a6764a5e024903848561c9935b1a43
I20260618 13:59:37.022845 29746 ts_tablet_manager.cc:1950] T 37a6764a5e024903848561c9935b1a43 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:37.023589 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 89) took 2461 ms. Trace:
I20260618 13:59:37.023725 29746 rpcz_store.cc:276] 0618 13:59:34.562462 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.562511 (+ 49us) service_pool.cc:224] Handling call
0618 13:59:36.989653 (+2427142us) ts_tablet_manager.cc:1151] Deleting tablet 37a6764a5e024903848561c9935b1a43
0618 13:59:36.989662 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.004770 (+ 15108us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.022292 (+ 17522us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.023564 (+ 1272us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2426934}
I20260618 13:59:37.024165 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 37a6764a5e024903848561c9935b1a43 (table default.Table_88951dd073b74c99aa4a0bd7dcf2e19d [id=51cf184bc7634141841001c082262441]) successfully deleted
I20260618 13:59:37.024984 29746 tablet_replica.cc:333] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:37.025182 29746 raft_consensus.cc:2243] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [term 2 LEADER]: Raft consensus shutting down.
I20260618 13:59:37.025405 29746 pending_rounds.cc:70] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2: Trying to abort 1 pending ops.
I20260618 13:59:37.025456 29746 pending_rounds.cc:77] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7298216658221109248 op_type: NO_OP noop_request { }
I20260618 13:59:37.025558 29746 raft_consensus.cc:2889] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:37.025643 29746 raft_consensus.cc:2272] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.026584 28799 tablet_bootstrap.cc:654] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.028208 29746 ts_tablet_manager.cc:1916] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.027318 29745 ts_tablet_manager.cc:1950] T e95c88e4b0a046a3ad69ee49af47c7c3 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:37.029717 30152 raft_consensus.cc:493] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.029838 30152 raft_consensus.cc:515] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.030364 30152 leader_election.cc:290] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:37.030959 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b21dcf74fa4264947ca43cc0026e76" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:37.031044 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b21dcf74fa4264947ca43cc0026e76" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:37.031515 28346 leader_election.cc:343] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:37.031708 28346 leader_election.cc:343] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.031788 28346 leader_election.cc:304] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.032090 30152 raft_consensus.cc:2749] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:37.036991 28618 consensus_peers.cc:597] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:37.038008 28797 raft_consensus.cc:740] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.039278 28798 tablet_bootstrap.cc:492] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:37.039415 28798 ts_tablet_manager.cc:1403] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.208s user 0.002s sys 0.002s
I20260618 13:59:37.040153 28798 raft_consensus.cc:359] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.040321 28798 raft_consensus.cc:385] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.040385 28798 raft_consensus.cc:740] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.040565 28798 consensus_queue.cc:260] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.041026 28798 ts_tablet_manager.cc:1434] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:37.041265 28798 tablet_bootstrap.cc:492] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:37.042496 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:37.042783 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:37.043162 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 69) took 4107 ms. Trace:
I20260618 13:59:37.043272 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:37.043287 29745 rpcz_store.cc:276] 0618 13:59:32.935889 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.936266 (+ 377us) service_pool.cc:224] Handling call
0618 13:59:36.967381 (+4031115us) ts_tablet_manager.cc:1151] Deleting tablet e95c88e4b0a046a3ad69ee49af47c7c3
0618 13:59:36.967394 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:36.971758 (+ 4364us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:36.974083 (+ 2325us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.043143 (+ 69060us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4024498}
I20260618 13:59:37.043905 29745 tablet_replica.cc:333] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:37.044143 29745 raft_consensus.cc:2243] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.044255 28797 consensus_queue.cc:260] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.044353 29745 raft_consensus.cc:2272] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.044907 29745 ts_tablet_manager.cc:1916] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.045001 28797 ts_tablet_manager.cc:1434] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.033s user 0.017s sys 0.001s
I20260618 13:59:37.045255 28797 tablet_bootstrap.cc:492] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:37.046556 28797 tablet_bootstrap.cc:654] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.046651 28404 raft_consensus.cc:2468] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:37.050880 28617 leader_election.cc:304] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.051208 29303 raft_consensus.cc:2804] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:37.047505 28799 tablet_bootstrap.cc:492] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:37.050117 30151 raft_consensus.cc:493] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.051311 29303 raft_consensus.cc:697] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:37.051422 28799 ts_tablet_manager.cc:1403] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.087s user 0.001s sys 0.003s
I20260618 13:59:37.051512 29303 consensus_queue.cc:237] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.051833 30151 raft_consensus.cc:515] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.053244 29853 ts_manager.cc:194] Registered new tserver with Master: d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:37.057802 28617 consensus_peers.cc:597] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4d6b01b897134d5bbd6a7d93b82ddb5e. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:37.065609 29853 ts_manager.cc:194] Registered new tserver with Master: 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:37.066931 28798 tablet_bootstrap.cc:654] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.043825 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet e95c88e4b0a046a3ad69ee49af47c7c3 (table default.Table_68ab7758379b44628a68285f209716ef [id=61ce11eead544e74a31db940c3376f4b]) successfully deleted
I20260618 13:59:37.043716 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:37.075567 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:37.075814 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:37.076007 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:37.076239 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_c70d6c5b0dc94b33b4c5772edd2b9dca [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 2 got 0
I20260618 13:59:37.076732 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:37.077016 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c881dd3cb66344969310956ad6e83cc0 (table default.Table_0158779f563348c78b3505f98d43b839 [id=b1d85ea3aa7047d285ab7c86da8213fd]). Expected version 1 got 0
I20260618 13:59:37.077229 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:37.077482 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:37.077759 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:37.078254 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:37.078711 28797 tablet_bootstrap.cc:492] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:37.078851 28797 ts_tablet_manager.cc:1403] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.034s user 0.000s sys 0.008s
I20260618 13:59:37.079247 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:37.079492 28797 raft_consensus.cc:359] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.079680 28797 raft_consensus.cc:385] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.079762 28797 raft_consensus.cc:740] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.079989 28797 consensus_queue.cc:260] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.080246 29746 ts_tablet_manager.cc:1929] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260618 13:59:37.080399 29746 log.cc:1199] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/b6dd422c05324c989e29b20ae2afabad
I20260618 13:59:37.080721 28797 ts_tablet_manager.cc:1434] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:37.081246 29746 ts_tablet_manager.cc:1950] T b6dd422c05324c989e29b20ae2afabad P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:37.081557 30151 leader_election.cc:290] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:37.082067 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 90) took 2510 ms. Trace:
I20260618 13:59:37.082137 29746 rpcz_store.cc:276] 0618 13:59:34.571390 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.571437 (+ 47us) service_pool.cc:224] Handling call
0618 13:59:37.024938 (+2453501us) ts_tablet_manager.cc:1151] Deleting tablet b6dd422c05324c989e29b20ae2afabad
0618 13:59:37.024948 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.031288 (+ 6340us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.080188 (+ 48900us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.082048 (+ 1860us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2453311}
I20260618 13:59:37.082147 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f45556a59c424ca2bc30956566374c89" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:37.082823 29746 tablet_replica.cc:333] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:37.082931 28797 tablet_bootstrap.cc:492] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:37.083024 29746 raft_consensus.cc:2243] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.083068 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f45556a59c424ca2bc30956566374c89" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:37.083161 29746 raft_consensus.cc:2272] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:37.083563 28346 leader_election.cc:343] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.083681 29746 ts_tablet_manager.cc:1916] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:37.083757 28346 leader_election.cc:343] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.083845 28346 leader_election.cc:304] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.084203 28797 tablet_bootstrap.cc:654] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.084389 30151 raft_consensus.cc:2749] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:37.085322 30151 raft_consensus.cc:493] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.085461 30151 raft_consensus.cc:515] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.086933 30151 leader_election.cc:290] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:37.087703 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f83bb8c3c504d5f9affbaa8be3f1508" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:37.088284 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f83bb8c3c504d5f9affbaa8be3f1508" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:37.089434 28346 leader_election.cc:343] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:37.089613 28346 leader_election.cc:343] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.089691 28346 leader_election.cc:304] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.090111 30151 raft_consensus.cc:2749] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:37.101118 28797 tablet_bootstrap.cc:492] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:37.101248 28797 ts_tablet_manager.cc:1403] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.004s
I20260618 13:59:37.101843 28799 raft_consensus.cc:359] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.101843 28797 raft_consensus.cc:359] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.101994 28799 raft_consensus.cc:385] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.102010 28797 raft_consensus.cc:385] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.102036 28799 raft_consensus.cc:740] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.102061 28797 raft_consensus.cc:740] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.102201 28799 consensus_queue.cc:260] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.102269 28797 consensus_queue.cc:260] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.102777 28798 tablet_bootstrap.cc:492] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:37.102922 28798 ts_tablet_manager.cc:1403] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.062s user 0.003s sys 0.002s
I20260618 13:59:37.103001 29566 ts_tablet_manager.cc:1929] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.103011 28797 ts_tablet_manager.cc:1434] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:37.103104 29566 log.cc:1199] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/fe750fc5d998448caeab0a6c808dfba7
I20260618 13:59:37.103293 28797 tablet_bootstrap.cc:492] T 650ed7ee8b134bc4b9f95028c54f99be P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:37.103456 29566 ts_tablet_manager.cc:1950] T fe750fc5d998448caeab0a6c808dfba7 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:37.104115 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 79) took 5010 ms. Trace:
I20260618 13:59:37.104188 29566 rpcz_store.cc:276] 0618 13:59:32.093737 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.093803 (+ 66us) service_pool.cc:224] Handling call
0618 13:59:37.021056 (+4927253us) ts_tablet_manager.cc:1151] Deleting tablet fe750fc5d998448caeab0a6c808dfba7
0618 13:59:37.021065 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.068893 (+ 47828us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.102950 (+ 34057us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.104096 (+ 1146us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4927042}
I20260618 13:59:37.104588 28797 tablet_bootstrap.cc:654] T 650ed7ee8b134bc4b9f95028c54f99be P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.104964 29759 raft_consensus.cc:493] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:37.105086 29759 raft_consensus.cc:515] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.105813 29759 leader_election.cc:290] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:37.106612 28484 leader_election.cc:343] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: d6871232860b41d489e34d95eb897241
W20260618 13:59:37.106709 28483 leader_election.cc:343] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: d6871232860b41d489e34d95eb897241
I20260618 13:59:37.106802 28483 leader_election.cc:304] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.107369 29759 raft_consensus.cc:2749] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:37.110360 28799 ts_tablet_manager.cc:1434] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.059s user 0.002s sys 0.000s
I20260618 13:59:37.110464 28798 raft_consensus.cc:359] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.110670 28799 tablet_bootstrap.cc:492] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:37.110663 28798 raft_consensus.cc:385] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.110723 28798 raft_consensus.cc:740] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.082615 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet b6dd422c05324c989e29b20ae2afabad (table default.Table_9424eec16a284672a0fe7ab3d0852b40 [id=070ea48af78346aa8ff43f2bb369730c]) successfully deleted
I20260618 13:59:37.111013 28798 consensus_queue.cc:260] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.111130 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet fe750fc5d998448caeab0a6c808dfba7 (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]) successfully deleted
I20260618 13:59:37.111506 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:37.111722 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:37.112610 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_7f787ab851be496dae97cba8835db19b [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 1 got 0
I20260618 13:59:37.113736 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:37.113785 29745 ts_tablet_manager.cc:1929] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.113878 29745 log.cc:1199] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/fe750fc5d998448caeab0a6c808dfba7
I20260618 13:59:37.114099 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:37.114840 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:37.118779 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:37.119484 28798 ts_tablet_manager.cc:1434] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.016s user 0.002s sys 0.000s
I20260618 13:59:37.119841 28798 tablet_bootstrap.cc:492] T b95bdb0e36584d92b9c28de257553827 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:37.123690 29745 ts_tablet_manager.cc:1950] T fe750fc5d998448caeab0a6c808dfba7 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:37.126645 30192 mvcc.cc:204] Tried to move back new op lower bound from 7298216647421165568 to 7298216623660199936. Current Snapshot: MvccSnapshot[applied={T|T < 7298216647421165568 or (T in {7298216647421165568})}]
I20260618 13:59:37.128438 28798 tablet_bootstrap.cc:654] T b95bdb0e36584d92b9c28de257553827 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.132937 29566 tablet_replica.cc:333] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:37.135876 29566 raft_consensus.cc:2243] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:37.136842 28799 tablet_bootstrap.cc:654] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.138073 29566 raft_consensus.cc:2272] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.138628 29566 ts_tablet_manager.cc:1916] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.139906 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 71) took 4203 ms. Trace:
I20260618 13:59:37.139987 29745 rpcz_store.cc:276] 0618 13:59:32.936580 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.936655 (+ 75us) service_pool.cc:224] Handling call
0618 13:59:37.043857 (+4107202us) ts_tablet_manager.cc:1151] Deleting tablet fe750fc5d998448caeab0a6c808dfba7
0618 13:59:37.043868 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.083620 (+ 39752us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.113720 (+ 30100us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.139861 (+ 26141us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4100967}
I20260618 13:59:37.140293 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 63) took 4205 ms. Trace:
I20260618 13:59:37.140360 29745 rpcz_store.cc:276] 0618 13:59:32.935204 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.936622 (+ 1418us) service_pool.cc:224] Handling call
0618 13:59:37.140219 (+4203597us) ts_tablet_manager.cc:1151] Deleting tablet 69d5e75b4a8247b29d69d07fad5ca5db
0618 13:59:37.140227 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.140281 (+ 54us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4197236}
I20260618 13:59:37.140616 29745 tablet_replica.cc:333] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:37.140771 29745 raft_consensus.cc:2243] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.140895 29745 raft_consensus.cc:2272] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.141322 29745 ts_tablet_manager.cc:1916] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.134413 28797 tablet_bootstrap.cc:492] T 650ed7ee8b134bc4b9f95028c54f99be P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:37.142953 28797 ts_tablet_manager.cc:1403] T 650ed7ee8b134bc4b9f95028c54f99be P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.040s user 0.004s sys 0.012s
I20260618 13:59:37.143600 28797 raft_consensus.cc:359] T 650ed7ee8b134bc4b9f95028c54f99be P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.143838 28797 raft_consensus.cc:385] T 650ed7ee8b134bc4b9f95028c54f99be P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.143903 28797 raft_consensus.cc:740] T 650ed7ee8b134bc4b9f95028c54f99be P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.144127 28797 consensus_queue.cc:260] T 650ed7ee8b134bc4b9f95028c54f99be P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.144619 28797 ts_tablet_manager.cc:1434] T 650ed7ee8b134bc4b9f95028c54f99be P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:37.142633 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet fe750fc5d998448caeab0a6c808dfba7 (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]) successfully deleted
W20260618 13:59:37.150708 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 69d5e75b4a8247b29d69d07fad5ca5db because the tablet was not found. No further retry: Not found: Tablet not found: 69d5e75b4a8247b29d69d07fad5ca5db
I20260618 13:59:37.151091 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:37.151470 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:37.152159 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 72c04f55890648f0a8184f3783243f7c (table default.Table_773be314d7e741fcb4fb6eb7d954802e [id=0d83fc68255c458587390c58b12e00d7]). Expected version 2 got 0
I20260618 13:59:37.152614 29745 ts_tablet_manager.cc:1929] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.153187 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
W20260618 13:59:37.153321 28617 consensus_peers.cc:597] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:37.153462 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a]). Expected version 1 got 0
I20260618 13:59:37.154004 28797 tablet_bootstrap.cc:492] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:37.155761 28797 tablet_bootstrap.cc:654] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.156332 29745 log.cc:1199] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/2f5a334d18ba4c06bb358fa86d8af80d
I20260618 13:59:37.156693 29745 ts_tablet_manager.cc:1950] T 2f5a334d18ba4c06bb358fa86d8af80d P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:37.168749 28798 tablet_bootstrap.cc:492] T b95bdb0e36584d92b9c28de257553827 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:37.169162 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 72) took 4232 ms. Trace:
I20260618 13:59:37.169855 29745 rpcz_store.cc:276] 0618 13:59:32.936802 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.936850 (+ 48us) service_pool.cc:224] Handling call
0618 13:59:37.140546 (+4203696us) ts_tablet_manager.cc:1151] Deleting tablet 2f5a334d18ba4c06bb358fa86d8af80d
0618 13:59:37.140585 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.147587 (+ 7002us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.152570 (+ 4983us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.169145 (+ 16575us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4197457}
I20260618 13:59:37.153847 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:37.174752 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:37.175279 29011 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:37.176923 29745 tablet_replica.cc:333] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:37.177030 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 2f5a334d18ba4c06bb358fa86d8af80d (table default.Table_2cb0539405824600bb10d2712fb17d31 [id=f90028f13e704830a408df1907fdae49]) successfully deleted
I20260618 13:59:37.177122 29745 raft_consensus.cc:2243] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.177263 29745 raft_consensus.cc:2272] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.177786 29745 ts_tablet_manager.cc:1916] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.179147 30151 raft_consensus.cc:493] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.179313 30151 raft_consensus.cc:515] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.180022 30151 leader_election.cc:290] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:37.180759 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed648dcf882340388155068d7a5a7869" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:37.181437 28346 leader_election.cc:343] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.182018 28798 ts_tablet_manager.cc:1403] T b95bdb0e36584d92b9c28de257553827 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.062s user 0.002s sys 0.001s
I20260618 13:59:37.170722 29746 ts_tablet_manager.cc:1929] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.182567 29746 log.cc:1199] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/4c2b924a096b479cad31c29bf7c99172
I20260618 13:59:37.182999 29746 ts_tablet_manager.cc:1950] T 4c2b924a096b479cad31c29bf7c99172 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:37.183816 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 91) took 2611 ms. Trace:
I20260618 13:59:37.183954 29746 rpcz_store.cc:276] 0618 13:59:34.572505 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.572559 (+ 54us) service_pool.cc:224] Handling call
0618 13:59:37.082778 (+2510219us) ts_tablet_manager.cc:1151] Deleting tablet 4c2b924a096b479cad31c29bf7c99172
0618 13:59:37.082787 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.139567 (+ 56780us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.170662 (+ 31095us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.183731 (+ 13069us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2509988}
I20260618 13:59:37.184506 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 4c2b924a096b479cad31c29bf7c99172 (table default.Table_3deacb2c9668491a85bbb09ae3f57d66 [id=431a660bfbed46248a603dc7781cf543]) successfully deleted
I20260618 13:59:37.184876 29746 tablet_replica.cc:333] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:37.185094 29746 raft_consensus.cc:2243] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.185142 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed648dcf882340388155068d7a5a7869" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:37.185313 29746 raft_consensus.cc:2272] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.185812 29746 ts_tablet_manager.cc:1916] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:37.186381 28346 leader_election.cc:343] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.186498 28346 leader_election.cc:304] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.187176 30151 raft_consensus.cc:2749] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:37.188268 28798 raft_consensus.cc:359] T b95bdb0e36584d92b9c28de257553827 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.188431 28798 raft_consensus.cc:385] T b95bdb0e36584d92b9c28de257553827 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.188491 28798 raft_consensus.cc:740] T b95bdb0e36584d92b9c28de257553827 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.188659 28798 consensus_queue.cc:260] T b95bdb0e36584d92b9c28de257553827 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.191574 28798 ts_tablet_manager.cc:1434] T b95bdb0e36584d92b9c28de257553827 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260618 13:59:37.191828 28798 tablet_bootstrap.cc:492] T 3a0039a463ec4a9c82a21a3a69b565fc P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:37.192786 28799 tablet_bootstrap.cc:492] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:37.192926 28799 ts_tablet_manager.cc:1403] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.082s user 0.009s sys 0.000s
I20260618 13:59:37.193717 28799 raft_consensus.cc:359] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.193905 28799 raft_consensus.cc:385] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.193979 28799 raft_consensus.cc:740] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.194164 28799 consensus_queue.cc:260] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.194686 28799 ts_tablet_manager.cc:1434] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:37.210678 29566 ts_tablet_manager.cc:1929] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.210878 29566 log.cc:1199] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/4d6b01b897134d5bbd6a7d93b82ddb5e
I20260618 13:59:37.211338 29566 ts_tablet_manager.cc:1950] T 4d6b01b897134d5bbd6a7d93b82ddb5e P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:37.212218 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 80) took 5115 ms. Trace:
I20260618 13:59:37.214582 28799 tablet_bootstrap.cc:492] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:37.214706 29566 rpcz_store.cc:276] 0618 13:59:32.096753 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.096810 (+ 57us) service_pool.cc:224] Handling call
0618 13:59:37.132882 (+5036072us) ts_tablet_manager.cc:1151] Deleting tablet 4d6b01b897134d5bbd6a7d93b82ddb5e
0618 13:59:37.132894 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.156482 (+ 23588us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.210609 (+ 54127us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.212200 (+ 1591us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5035877}
I20260618 13:59:37.215260 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 4d6b01b897134d5bbd6a7d93b82ddb5e (table default.Table_45297e5c8f754fbbb3ab4dac6de1e136 [id=8c16c31ead484af3a1f3ee482b4cc233]) successfully deleted
I20260618 13:59:37.215698 29566 tablet_replica.cc:333] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:37.215931 29566 raft_consensus.cc:2243] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:37.216296 29566 raft_consensus.cc:2272] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.217064 29566 ts_tablet_manager.cc:1916] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.217098 28799 tablet_bootstrap.cc:654] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.223483 28797 tablet_bootstrap.cc:492] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:37.223613 28797 ts_tablet_manager.cc:1403] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.070s user 0.007s sys 0.009s
I20260618 13:59:37.224313 28797 raft_consensus.cc:359] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.224516 28797 raft_consensus.cc:385] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.224584 28797 raft_consensus.cc:740] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.224745 28798 tablet_bootstrap.cc:654] T 3a0039a463ec4a9c82a21a3a69b565fc P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.224824 28797 consensus_queue.cc:260] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.225418 28797 ts_tablet_manager.cc:1434] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:37.225673 28797 tablet_bootstrap.cc:492] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:37.227202 28797 tablet_bootstrap.cc:654] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.239418 28799 tablet_bootstrap.cc:492] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:37.239257 29746 ts_tablet_manager.cc:1929] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.240427 29746 log.cc:1199] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/dd2385e6752d49429422fc854539e9c6
I20260618 13:59:37.240849 29746 ts_tablet_manager.cc:1950] T dd2385e6752d49429422fc854539e9c6 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:37.241976 28799 ts_tablet_manager.cc:1403] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.027s user 0.004s sys 0.000s
I20260618 13:59:37.242671 28799 raft_consensus.cc:359] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.242826 28799 raft_consensus.cc:385] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.242873 28799 raft_consensus.cc:740] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.243325 28799 consensus_queue.cc:260] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.243852 28799 ts_tablet_manager.cc:1434] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:37.246562 28797 tablet_bootstrap.cc:492] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:37.246683 28797 ts_tablet_manager.cc:1403] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.000s
I20260618 13:59:37.247363 28797 raft_consensus.cc:359] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.247539 28797 raft_consensus.cc:385] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.247597 28797 raft_consensus.cc:740] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.247774 28797 consensus_queue.cc:260] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.248257 28797 ts_tablet_manager.cc:1434] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:37.248521 28797 tablet_bootstrap.cc:492] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:36.903476 29894 rpcz_store.cc:276] 0618 13:59:34.075168 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.075229 (+ 61us) service_pool.cc:224] Handling call
0618 13:59:35.963405 (+1888176us) raft_consensus.cc:1471] Updating replica for 84 ops
0618 13:59:36.365332 (+401927us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.365339 (+ 7us) write_op.cc:454] Released schema lock
0618 13:59:36.365780 (+ 441us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.365784 (+ 4us) write_op.cc:454] Released schema lock
0618 13:59:36.411243 (+ 45459us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.411251 (+ 8us) write_op.cc:454] Released schema lock
0618 13:59:36.412166 (+ 915us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.412170 (+ 4us) write_op.cc:454] Released schema lock
0618 13:59:36.412545 (+ 375us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.412549 (+ 4us) write_op.cc:454] Released schema lock
0618 13:59:36.412975 (+ 426us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.412980 (+ 5us) write_op.cc:454] Released schema lock
0618 13:59:36.413318 (+ 338us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.413347 (+ 29us) write_op.cc:454] Released schema lock
0618 13:59:36.414001 (+ 654us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.414006 (+ 5us) write_op.cc:454] Released schema lock
0618 13:59:36.414329 (+ 323us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.414334 (+ 5us) write_op.cc:454] Released schema lock
0618 13:59:36.556041 (+141707us) raft_consensus.cc:1534] Early marking committed up to index 0
0618 13:59:36.556049 (+ 8us) raft_consensus.cc:1539] Triggering prepare for 84 ops
0618 13:59:36.556518 (+ 469us) write_op.cc:270] Start()
0618 13:59:36.556581 (+ 63us) write_op.cc:276] Timestamp: P: 1781791169818825 usec, L: 0
0618 13:59:36.579051 (+ 22470us) write_op.cc:270] Start()
0618 13:59:36.579092 (+ 41us) write_op.cc:276] Timestamp: P: 1781791169847623 usec, L: 0
0618 13:59:36.579298 (+ 206us) write_op.cc:270] Start()
0618 13:59:36.579320 (+ 22us) write_op.cc:276] Timestamp: P: 1781791169849182 usec, L: 0
0618 13:59:36.579449 (+ 129us) write_op.cc:270] Start()
0618 13:59:36.579467 (+ 18us) write_op.cc:276] Timestamp: P: 1781791169854090 usec, L: 0
0618 13:59:36.579596 (+ 129us) write_op.cc:270] Start()
0618 13:59:36.579614 (+ 18us) write_op.cc:276] Timestamp: P: 1781791169857863 usec, L: 0
0618 13:59:36.579727 (+ 113us) write_op.cc:270] Start()
0618 13:59:36.579744 (+ 17us) write_op.cc:276] Timestamp: P: 1781791169861128 usec, L: 0
0618 13:59:36.579882 (+ 138us) write_op.cc:270] Start()
0618 13:59:36.579899 (+ 17us) write_op.cc:276] Timestamp: P: 1781791169863741 usec, L: 0
0618 13:59:36.580011 (+ 112us) write_op.cc:270] Start()
0618 13:59:36.580029 (+ 18us) write_op.cc:276] Timestamp: P: 1781791169887647 usec, L: 0
0618 13:59:36.580160 (+ 131us) write_op.cc:270] Start()
0618 13:59:36.580177 (+ 17us) write_op.cc:276] Timestamp: P: 1781791169898858 usec, L: 0
0618 13:59:36.580297 (+ 120us) write_op.cc:270] Start()
0618 13:59:36.580317 (+ 20us) write_op.cc:276] Timestamp: P: 1781791170091042 usec, L: 0
0618 13:59:36.580435 (+ 118us) write_op.cc:270] Start()
0618 13:59:36.580452 (+ 17us) write_op.cc:276] Timestamp: P: 1781791170093107 usec, L: 0
0618 13:59:36.580568 (+ 116us) write_op.cc:270] Start()
0618 13:59:36.580601 (+ 33us) write_op.cc:276] Timestamp: P: 1781791170099055 usec, L: 0
0618 13:59:36.580723 (+ 122us) write_op.cc:270] Start()
0618 13:59:36.580741 (+ 18us) write_op.cc:276] Timestamp: P: 1781791170099709 usec, L: 0
0618 13:59:36.580885 (+ 144us) write_op.cc:270] Start()
0618 13:59:36.580902 (+ 17us) write_op.cc:276] Timestamp: P: 1781791170140602 usec, L: 0
0618 13:59:36.581045 (+ 143us) write_op.cc:270] Start()
0618 13:59:36.581064 (+ 19us) write_op.cc:276] Timestamp: P: 1781791170245500 usec, L: 0
0618 13:59:36.581189 (+ 125us) write_op.cc:270] Start()
0618 13:59:36.581207 (+ 18us) write_op.cc:276] Timestamp: P: 1781791170293162 usec, L: 0
0618 13:59:36.581346 (+ 139us) write_op.cc:270] Start()
0618 13:59:36.581372 (+ 26us) write_op.cc:276] Timestamp: P: 1781791170340714 usec, L: 0
0618 13:59:36.581512 (+ 140us) write_op.cc:270] Start()
0618 13:59:36.581531 (+ 19us) write_op.cc:276] Timestamp: P: 1781791170351249 usec, L: 0
0618 13:59:36.581659 (+ 128us) write_op.cc:270] Start()
0618 13:59:36.581676 (+ 17us) write_op.cc:276] Timestamp: P: 1781791170351916 usec, L: 0
0618 13:59:36.581828 (+ 152us) write_op.cc:270] Start()
0618 13:59:36.581848 (+ 20us) write_op.cc:276] Timestamp: P: 1781791170397924 usec, L: 0
0618 13:59:36.581971 (+ 123us) write_op.cc:270] Start()
0618 13:59:36.581988 (+ 17us) write_op.cc:276] Timestamp: P: 1781791170398588 usec, L: 0
0618 13:59:36.582105 (+ 117us) write_op.cc:270] Start()
0618 13:59:36.582123 (+ 18us) write_op.cc:276] Timestamp: P: 1781791170416513 usec, L: 0
0618 13:59:36.582266 (+ 143us) write_op.cc:270] Start()
0618 13:59:36.582284 (+ 18us) write_op.cc:276] Timestamp: P: 1781791170417224 usec, L: 0
0618 13:59:36.597447 (+ 15163us) write_op.cc:270] Start()
0618 13:59:36.597805 (+ 358us) write_op.cc:276] Timestamp: P: 1781791170418134 usec, L: 0
0618 13:59:36.598035 (+ 230us) write_op.cc:270] Start()
0618 13:59:36.598061 (+ 26us) write_op.cc:276] Timestamp: P: 1781791170418759 usec, L: 0
0618 13:59:36.598369 (+ 308us) write_op.cc:270] Start()
0618 13:59:36.598397 (+ 28us) write_op.cc:276] Timestamp: P: 1781791170533113 usec, L: 0
0618 13:59:36.598583 (+ 186us) write_op.cc:270] Start()
0618 13:59:36.598609 (+ 26us) write_op.cc:276] Timestamp: P: 1781791170535018 usec, L: 0
0618 13:59:36.598753 (+ 144us) write_op.cc:270] Start()
0618 13:59:36.598785 (+ 32us) write_op.cc:276] Timestamp: P: 1781791170583964 usec, L: 0
0618 13:59:36.598914 (+ 129us) write_op.cc:270] Start()
0618 13:59:36.598935 (+ 21us) write_op.cc:276] Timestamp: P: 1781791170585299 usec, L: 0
0618 13:59:36.599053 (+ 118us) write_op.cc:270] Start()
0618 13:59:36.599071 (+ 18us) write_op.cc:276] Timestamp: P: 1781791170717965 usec, L: 0
0618 13:59:36.599179 (+ 108us) write_op.cc:270] Start()
0618 13:59:36.599196 (+ 17us) write_op.cc:276] Timestamp: P: 1781791170738750 usec, L: 0
0618 13:59:36.599337 (+ 141us) write_op.cc:270] Start()
0618 13:59:36.599357 (+ 20us) write_op.cc:276] Timestamp: P: 1781791170749800 usec, L: 0
0618 13:59:36.599498 (+ 141us) write_op.cc:270] Start()
0618 13:59:36.599517 (+ 19us) write_op.cc:276] Timestamp: P: 1781791170769773 usec, L: 0
0618 13:59:36.599665 (+ 148us) write_op.cc:270] Start()
0618 13:59:36.599688 (+ 23us) write_op.cc:276] Timestamp: P: 1781791170773971 usec, L: 0
0618 13:59:36.599838 (+ 150us) write_op.cc:270] Start()
0618 13:59:36.599858 (+ 20us) write_op.cc:276] Timestamp: P: 1781791170886455 usec, L: 0
0618 13:59:36.600013 (+ 155us) write_op.cc:270] Start()
0618 13:59:36.600048 (+ 35us) write_op.cc:276] Timestamp: P: 1781791170928047 usec, L: 0
0618 13:59:36.600196 (+ 148us) write_op.cc:270] Start()
0618 13:59:36.600237 (+ 41us) write_op.cc:276] Timestamp: P: 1781791171083614 usec, L: 0
0618 13:59:36.600393 (+ 156us) write_op.cc:270] Start()
0618 13:59:36.600416 (+ 23us) write_op.cc:276] Timestamp: P: 1781791171277871 usec, L: 0
0618 13:59:36.600559 (+ 143us) write_op.cc:270] Start()
0618 13:59:36.600579 (+ 20us) write_op.cc:276] Timestamp: P: 1781791171390589 usec, L: 0
0618 13:59:36.600725 (+ 146us) write_op.cc:270] Start()
0618 13:59:36.600746 (+ 21us) write_op.cc:276] Timestamp: P: 1781791171661649 usec, L: 0
0618 13:59:36.601202 (+ 456us) write_op.cc:270] Start()
0618 13:59:36.601229 (+ 27us) write_op.cc:276] Timestamp: P: 1781791171931327 usec, L: 0
0618 13:59:36.601463 (+ 234us) write_op.cc:270] Start()
0618 13:59:36.601488 (+ 25us) write_op.cc:276] Timestamp: P: 1781791171942164 usec, L: 0
0618 13:59:36.607681 (+ 6193us) write_op.cc:270] Start()
0618 13:59:36.607714 (+ 33us) write_op.cc:276] Timestamp: P: 1781791171951416 usec, L: 0
0618 13:59:36.607893 (+ 179us) write_op.cc:270] Start()
0618 13:59:36.607913 (+ 20us) write_op.cc:276] Timestamp: P: 1781791172000059 usec, L: 0
0618 13:59:36.608066 (+ 153us) write_op.cc:270] Start()
0618 13:59:36.608085 (+ 19us) write_op.cc:276] Timestamp: P: 1781791172080326 usec, L: 0
0618 13:59:36.608212 (+ 127us) write_op.cc:270] Start()
0618 13:59:36.608235 (+ 23us) write_op.cc:276] Timestamp: P: 1781791172204925 usec, L: 0
0618 13:59:36.609113 (+ 878us) write_op.cc:270] Start()
0618 13:59:36.609136 (+ 23us) write_op.cc:276] Timestamp: P: 1781791172206366 usec, L: 0
0618 13:59:36.609339 (+ 203us) write_op.cc:270] Start()
0618 13:59:36.609377 (+ 38us) write_op.cc:276] Timestamp: P: 1781791172217417 usec, L: 0
0618 13:59:36.609507 (+ 130us) write_op.cc:270] Start()
0618 13:59:36.609525 (+ 18us) write_op.cc:276] Timestamp: P: 1781791172278247 usec, L: 0
0618 13:59:36.609647 (+ 122us) write_op.cc:270] Start()
0618 13:59:36.609685 (+ 38us) write_op.cc:276] Timestamp: P: 1781791172280582 usec, L: 0
0618 13:59:36.609819 (+ 134us) write_op.cc:270] Start()
0618 13:59:36.609841 (+ 22us) write_op.cc:276] Timestamp: P: 1781791172281669 usec, L: 0
0618 13:59:36.610008 (+ 167us) write_op.cc:270] Start()
0618 13:59:36.610032 (+ 24us) write_op.cc:276] Timestamp: P: 1781791172395297 usec, L: 0
0618 13:59:36.610177 (+ 145us) write_op.cc:270] Start()
0618 13:59:36.610196 (+ 19us) write_op.cc:276] Timestamp: P: 1781791172396016 usec, L: 0
0618 13:59:36.610311 (+ 115us) write_op.cc:270] Start()
0618 13:59:36.610328 (+ 17us) write_op.cc:276] Timestamp: P: 1781791172424508 usec, L: 0
0618 13:59:36.610458 (+ 130us) write_op.cc:270] Start()
0618 13:59:36.610479 (+ 21us) write_op.cc:276] Timestamp: P: 1781791172460615 usec, L: 0
0618 13:59:36.610616 (+ 137us) write_op.cc:270] Start()
0618 13:59:36.610636 (+ 20us) write_op.cc:276] Timestamp: P: 1781791172477506 usec, L: 0
0618 13:59:36.610791 (+ 155us) write_op.cc:270] Start()
0618 13:59:36.610815 (+ 24us) write_op.cc:276] Timestamp: P: 1781791172487535 usec, L: 0
0618 13:59:36.610964 (+ 149us) write_op.cc:270] Start()
0618 13:59:36.610983 (+ 19us) write_op.cc:276] Timestamp: P: 1781791172535386 usec, L: 0
0618 13:59:36.611113 (+ 130us) write_op.cc:270] Start()
0618 13:59:36.611133 (+ 20us) write_op.cc:276] Timestamp: P: 1781791172622680 usec, L: 0
0618 13:59:36.690671 (+ 79538us) write_op.cc:270] Start()
0618 13:59:36.690715 (+ 44us) write_op.cc:276] Timestamp: P: 1781791172678646 usec, L: 0
0618 13:59:36.690975 (+ 260us) write_op.cc:270] Start()
0618 13:59:36.691010 (+ 35us) write_op.cc:276] Timestamp: P: 1781791172870808 usec, L: 0
0618 13:59:36.691392 (+ 382us) write_op.cc:270] Start()
0618 13:59:36.691421 (+ 29us) write_op.cc:276] Timestamp: P: 1781791172903048 usec, L: 0
0618 13:59:36.691614 (+ 193us) write_op.cc:270] Start()
0618 13:59:36.691638 (+ 24us) write_op.cc:276] Timestamp: P: 1781791172911633 usec, L: 0
0618 13:59:36.691855 (+ 217us) write_op.cc:270] Start()
0618 13:59:36.691880 (+ 25us) write_op.cc:276] Timestamp: P: 1781791172930200 usec, L: 0
0618 13:59:36.692098 (+ 218us) write_op.cc:270] Start()
0618 13:59:36.692124 (+ 26us) write_op.cc:276] Timestamp: P: 1781791172959772 usec, L: 0
0618 13:59:36.692289 (+ 165us) write_op.cc:270] Start()
0618 13:59:36.692314 (+ 25us) write_op.cc:276] Timestamp: P: 1781791172984163 usec, L: 0
0618 13:59:36.692492 (+ 178us) write_op.cc:270] Start()
0618 13:59:36.692517 (+ 25us) write_op.cc:276] Timestamp: P: 1781791172985424 usec, L: 0
0618 13:59:36.692754 (+ 237us) write_op.cc:270] Start()
0618 13:59:36.692792 (+ 38us) write_op.cc:276] Timestamp: P: 1781791172986005 usec, L: 0
0618 13:59:36.692950 (+ 158us) write_op.cc:270] Start()
0618 13:59:36.692974 (+ 24us) write_op.cc:276] Timestamp: P: 1781791172986950 usec, L: 0
0618 13:59:36.693119 (+ 145us) write_op.cc:270] Start()
0618 13:59:36.693142 (+ 23us) write_op.cc:276] Timestamp: P: 1781791172987848 usec, L: 0
0618 13:59:36.693290 (+ 148us) write_op.cc:270] Start()
0618 13:59:36.693313 (+ 23us) write_op.cc:276] Timestamp: P: 1781791173153950 usec, L: 0
0618 13:59:36.693457 (+ 144us) write_op.cc:270] Start()
0618 13:59:36.693484 (+ 27us) write_op.cc:276] Timestamp: P: 1781791173154697 usec, L: 0
0618 13:59:36.693635 (+ 151us) write_op.cc:270] Start()
0618 13:59:36.693658 (+ 23us) write_op.cc:276] Timestamp: P: 1781791173163114 usec, L: 0
0618 13:59:36.693869 (+ 211us) write_op.cc:270] Start()
0618 13:59:36.693894 (+ 25us) write_op.cc:276] Timestamp: P: 1781791173163642 usec, L: 0
0618 13:59:36.694062 (+ 168us) write_op.cc:270] Start()
0618 13:59:36.694085 (+ 23us) write_op.cc:276] Timestamp: P: 1781791173180742 usec, L: 0
0618 13:59:36.694225 (+ 140us) write_op.cc:270] Start()
0618 13:59:36.694248 (+ 23us) write_op.cc:276] Timestamp: P: 1781791173208890 usec, L: 0
0618 13:59:36.694405 (+ 157us) write_op.cc:270] Start()
0618 13:59:36.694448 (+ 43us) write_op.cc:276] Timestamp: P: 1781791173272368 usec, L: 0
0618 13:59:36.694655 (+ 207us) write_op.cc:270] Start()
0618 13:59:36.694680 (+ 25us) write_op.cc:276] Timestamp: P: 1781791173295532 usec, L: 0
0618 13:59:36.694852 (+ 172us) write_op.cc:270] Start()
0618 13:59:36.694901 (+ 49us) write_op.cc:276] Timestamp: P: 1781791173300517 usec, L: 0
0618 13:59:36.695079 (+ 178us) write_op.cc:270] Start()
0618 13:59:36.695103 (+ 24us) write_op.cc:276] Timestamp: P: 1781791173418945 usec, L: 0
0618 13:59:36.753462 (+ 58359us) write_op.cc:270] Start()
0618 13:59:36.753520 (+ 58us) write_op.cc:276] Timestamp: P: 1781791173465701 usec, L: 0
0618 13:59:36.753802 (+ 282us) write_op.cc:270] Start()
0618 13:59:36.753827 (+ 25us) write_op.cc:276] Timestamp: P: 1781791173668356 usec, L: 0
0618 13:59:36.753977 (+ 150us) write_op.cc:270] Start()
0618 13:59:36.753995 (+ 18us) write_op.cc:276] Timestamp: P: 1781791173686808 usec, L: 0
0618 13:59:36.756002 (+ 2007us) log.cc:844] Serialized 59892 byte log entry
0618 13:59:36.853890 (+ 97888us) raft_consensus.cc:1645] Marking committed up to 584
0618 13:59:36.854069 (+ 179us) log.cc:844] Serialized 15 byte log entry
0618 13:59:36.866663 (+ 12594us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.866702 (+ 39us) write_op.cc:454] Released schema lock
0618 13:59:36.868501 (+ 1799us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.868506 (+ 5us) write_op.cc:454] Released schema lock
0618 13:59:36.881118 (+ 12612us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.881126 (+ 8us) write_op.cc:454] Released schema lock
0618 13:59:36.887805 (+ 6679us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.887809 (+ 4us) write_op.cc:454] Released schema lock
0618 13:59:36.889659 (+ 1850us) raft_consensus.cc:1697] Filling consensus response to leader.
0618 13:59:36.890804 (+ 1145us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x616000031054. stack: 00007fe995e05496 00007fe995e05250 00007fe994e303ad 000000000099c914 000000000099c8dd 00007fe9a52e66bd 00007fe9a52e67b3 00007fe9a1eaeb21 00007fe9a9e09fc5 00007fe9a9e1de25 00007fe9a9e1dbd1 00007fe9a322834d 00007fe9a1ebf92e 00007fe9a1ebf67d 0000000000aa681c 00007fe995e4b9ff
0618 13:59:36.892253 (+ 1449us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0618 13:59:36.892288 (+ 35us) raft_consensus.cc:1685] finished
0618 13:59:36.892291 (+ 3us) raft_consensus.cc:1691] UpdateReplicas() finished
0618 13:59:36.892445 (+ 154us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:36.566975 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:36.567144 (+ 169us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:36.567154 (+ 10us) write_op.cc:435] Acquired schema lock
0618 13:59:36.567157 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:36.567336 (+ 179us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:36.567358 (+ 22us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:36.567365 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:36.567463 (+ 98us) tablet.cc:701] Row locks acquired
0618 13:59:36.567467 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:36.863687 (+296220us) op_driver.cc:464] REPLICATION: finished
0618 13:59:36.864237 (+ 550us) write_op.cc:301] APPLY: starting
0618 13:59:36.864258 (+ 21us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:36.864581 (+ 323us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:36.864583 (+ 2us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:36.864794 (+ 211us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:36.865531 (+ 737us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:36.865586 (+ 55us) write_op.cc:312] APPLY: finished
0618 13:59:36.865812 (+ 226us) log.cc:844] Serialized 29 byte log entry
0618 13:59:36.865971 (+ 159us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.865995 (+ 24us) write_op.cc:454] Released schema lock
0618 13:59:36.866008 (+ 13us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0618 13:59:36.584560 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:36.584694 (+ 134us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:36.584701 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:36.584704 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:36.584844 (+ 140us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:36.584860 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:36.584865 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:36.584951 (+ 86us) tablet.cc:701] Row locks acquired
0618 13:59:36.584954 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:36.866831 (+281877us) op_driver.cc:464] REPLICATION: finished
0618 13:59:36.867086 (+ 255us) write_op.cc:301] APPLY: starting
0618 13:59:36.867098 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:36.867138 (+ 40us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:36.867142 (+ 4us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:36.867228 (+ 86us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:36.867295 (+ 67us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:36.867326 (+ 31us) write_op.cc:312] APPLY: finished
0618 13:59:36.867394 (+ 68us) log.cc:844] Serialized 29 byte log entry
0618 13:59:36.867461 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.867476 (+ 15us) write_op.cc:454] Released schema lock
0618 13:59:36.867483 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:36.867532 (+ 49us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.867536 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:36.585018 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:36.585107 (+ 89us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:36.585112 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:36.585114 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:36.585178 (+ 64us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:36.585189 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:36.585193 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:36.585246 (+ 53us) tablet.cc:701] Row locks acquired
0618 13:59:36.585248 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:36.866905 (+281657us) op_driver.cc:464] REPLICATION: finished
0618 13:59:36.867675 (+ 770us) write_op.cc:301] APPLY: starting
0618 13:59:36.867690 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:36.867724 (+ 34us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:36.867726 (+ 2us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:36.867818 (+ 92us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:36.867845 (+ 27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:36.867869 (+ 24us) write_op.cc:312] APPLY: finished
0618 13:59:36.867925 (+ 56us) log.cc:844] Serialized 29 byte log entry
0618 13:59:36.867971 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.867984 (+ 13us) write_op.cc:454] Released schema lock
0618 13:59:36.867989 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0618 13:59:36.585302 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:36.585381 (+ 79us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:36.585385 (+ 4us) write_op.cc:435] Acquired schema lock
0618 13:59:36.585387 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:36.585463 (+ 76us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:36.585473 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:36.585477 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:36.585596 (+ 119us) tablet.cc:701] Row locks acquired
0618 13:59:36.585599 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:36.868572 (+282973us) op_driver.cc:464] REPLICATION: finished
0618 13:59:36.868807 (+ 235us) write_op.cc:301] APPLY: starting
0618 13:59:36.868818 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:36.868846 (+ 28us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:36.868849 (+ 3us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:36.868897 (+ 48us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:36.868919 (+ 22us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:36.868950 (+ 31us) write_op.cc:312] APPLY: finished
0618 13:59:36.868998 (+ 48us) log.cc:844] Serialized 29 byte log entry
0618 13:59:36.869048 (+ 50us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.869061 (+ 13us) write_op.cc:454] Released schema lock
0618 13:59:36.869066 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:36.869105 (+ 39us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.869109 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:36.585653 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:36.585737 (+ 84us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:36.585741 (+ 4us) write_op.cc:435] Acquired schema lock
0618 13:59:36.585745 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:36.585819 (+ 74us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:36.585830 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:36.585834 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:36.585894 (+ 60us) tablet.cc:701] Row locks acquired
0618 13:59:36.585896 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:36.868627 (+282731us) op_driver.cc:464] REPLICATION: finished
0618 13:59:36.869160 (+ 533us) write_op.cc:301] APPLY: starting
0618 13:59:36.869168 (+ 8us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:36.869192 (+ 24us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:36.869195 (+ 3us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:36.869265 (+ 70us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:36.869299 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:36.869320 (+ 21us) write_op.cc:312] APPLY: finished
0618 13:59:36.869364 (+ 44us) log.cc:844] Serialized 29 byte log entry
0618 13:59:36.869406 (+ 42us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.869418 (+ 12us) write_op.cc:454] Released schema lock
0618 13:59:36.869423 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:36.869463 (+ 40us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.869466 (+ 3us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:36.585968 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:36.586060 (+ 92us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:36.586065 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:36.586067 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:36.586127 (+ 60us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:36.586138 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:36.586143 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:36.586188 (+ 45us) tablet.cc:701] Row locks acquired
0618 13:59:36.586191 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:36.868664 (+282473us) op_driver.cc:464] REPLICATION: finished
0618 13:59:36.869525 (+ 861us) write_op.cc:301] APPLY: starting
0618 13:59:36.869533 (+ 8us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:36.869557 (+ 24us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:36.869560 (+ 3us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:36.869588 (+ 28us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:36.869608 (+ 20us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:36.869642 (+ 34us) write_op.cc:312] APPLY: finished
0618 13:59:36.869688 (+ 46us) log.cc:844] Serialized 23 byte log entry
0618 13:59:36.869729 (+ 41us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.869739 (+ 10us) write_op.cc:454] Released schema lock
0618 13:59:36.869744 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0618 13:59:36.586259 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:36.586333 (+ 74us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:36.586337 (+ 4us) write_op.cc:435] Acquired schema lock
0618 13:59:36.586341 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:36.586409 (+ 68us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:36.586448 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:36.586453 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:36.586508 (+ 55us) tablet.cc:701] Row locks acquired
0618 13:59:36.586510 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:36.881214 (+294704us) op_driver.cc:464] REPLICATION: finished
0618 13:59:36.881540 (+ 326us) write_op.cc:301] APPLY: starting
0618 13:59:36.881554 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:36.881593 (+ 39us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:36.881596 (+ 3us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:36.881699 (+ 103us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:36.881767 (+ 68us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:36.881791 (+ 24us) write_op.cc:312] APPLY: finished
0618 13:59:36.881868 (+ 77us) log.cc:844] Serialized 29 byte log entry
0618 13:59:36.881939 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.881956 (+ 17us) write_op.cc:454] Released schema lock
0618 13:59:36.881963 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:36.882016 (+ 53us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.882020 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:36.586568 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:36.586666 (+ 98us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:36.586670 (+ 4us) write_op.cc:435] Acquired schema lock
0618 13:59:36.586672 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:36.586732 (+ 60us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:36.586741 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:36.586745 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:36.586805 (+ 60us) tablet.cc:701] Row locks acquired
0618 13:59:36.586807 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:36.881291 (+294484us) op_driver.cc:464] REPLICATION: finished
0618 13:59:36.882089 (+ 798us) write_op.cc:301] APPLY: starting
0618 13:59:36.882102 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:36.882567 (+ 465us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:36.882572 (+ 5us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:36.882685 (+ 113us) tablet.cc:1383] finished ApplyRowOperation cycle
0618
I20260618 13:59:37.255615 28799 tablet_bootstrap.cc:492] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:37.256217 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 92) took 2682 ms. Trace:
I20260618 13:59:37.257795 28799 tablet_bootstrap.cc:654] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.258249 30202 sys_catalog.cc:455] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:37.258555 30202 sys_catalog.cc:458] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: This master's current role is: LEADER
I20260618 13:59:37.241817 29745 ts_tablet_manager.cc:1929] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:37.259311 29745 log.cc:1199] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/7aa9f65d8a5c46fdb745ee17af946dce
I20260618 13:59:37.259639 29745 ts_tablet_manager.cc:1950] T 7aa9f65d8a5c46fdb745ee17af946dce P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:37.264958 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:37.265695 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 65) took 4330 ms. Trace:
I20260618 13:59:37.266894 28797 tablet_bootstrap.cc:654] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.267040 29745 rpcz_store.cc:276] 0618 13:59:32.935445 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.936229 (+ 784us) service_pool.cc:224] Handling call
0618 13:59:37.176851 (+4240622us) ts_tablet_manager.cc:1151] Deleting tablet 7aa9f65d8a5c46fdb745ee17af946dce
0618 13:59:37.176865 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.180718 (+ 3853us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.241766 (+ 61048us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.265680 (+ 23914us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4233297}
I20260618 13:59:37.267627 29745 tablet_replica.cc:333] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:37.267704 29746 rpcz_store.cc:276] 0618 13:59:34.573273 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.573344 (+ 71us) service_pool.cc:224] Handling call
0618 13:59:37.184820 (+2611476us) ts_tablet_manager.cc:1151] Deleting tablet dd2385e6752d49429422fc854539e9c6
0618 13:59:37.184830 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.188848 (+ 4018us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.239192 (+ 50344us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.256193 (+ 17001us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2611234}
I20260618 13:59:37.268504 29746 tablet_replica.cc:333] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:37.268726 29746 raft_consensus.cc:2243] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.268920 29746 raft_consensus.cc:2272] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:37.268962 28617 consensus_peers.cc:597] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:37.269568 29746 ts_tablet_manager.cc:1916] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.267638 30151 raft_consensus.cc:493] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.271376 30151 raft_consensus.cc:515] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.271940 30151 leader_election.cc:290] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:37.273255 28618 consensus_peers.cc:597] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:37.273248 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01c8463055e4526b3c75190066f30b6" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:37.273309 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01c8463055e4526b3c75190066f30b6" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:37.273715 28346 leader_election.cc:343] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:37.273916 28346 leader_election.cc:343] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.274021 28346 leader_election.cc:304] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.274796 30151 raft_consensus.cc:2749] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:37.277972 29745 raft_consensus.cc:2243] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.278184 29745 raft_consensus.cc:2272] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.259821 28798 tablet_bootstrap.cc:492] T 3a0039a463ec4a9c82a21a3a69b565fc P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:37.278764 29745 ts_tablet_manager.cc:1916] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.278798 28798 ts_tablet_manager.cc:1403] T 3a0039a463ec4a9c82a21a3a69b565fc P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.087s user 0.003s sys 0.001s
I20260618 13:59:37.279673 28798 raft_consensus.cc:359] T 3a0039a463ec4a9c82a21a3a69b565fc P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.279920 28798 raft_consensus.cc:385] T 3a0039a463ec4a9c82a21a3a69b565fc P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.280031 28798 raft_consensus.cc:740] T 3a0039a463ec4a9c82a21a3a69b565fc P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.280087 29566 ts_tablet_manager.cc:1929] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.280192 29566 log.cc:1199] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/b51d0851efb34eaa9e9e4be6b292ef62
I20260618 13:59:37.280403 28798 consensus_queue.cc:260] T 3a0039a463ec4a9c82a21a3a69b565fc P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.280572 29566 ts_tablet_manager.cc:1950] T b51d0851efb34eaa9e9e4be6b292ef62 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:37.281018 28798 ts_tablet_manager.cc:1434] T 3a0039a463ec4a9c82a21a3a69b565fc P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:37.281283 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 81) took 5183 ms. Trace:
I20260618 13:59:37.281812 28798 tablet_bootstrap.cc:492] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:37.283751 28798 tablet_bootstrap.cc:654] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.285074 29745 ts_tablet_manager.cc:1929] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.285173 29745 log.cc:1199] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/d7a97176f72349cda221baaea61d18ad
I20260618 13:59:37.285501 29745 ts_tablet_manager.cc:1950] T d7a97176f72349cda221baaea61d18ad P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:37.286168 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 76) took 4342 ms. Trace:
I20260618 13:59:37.293481 29749 raft_consensus.cc:493] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:37.293622 29749 raft_consensus.cc:515] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.294402 29749 leader_election.cc:290] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:37.294695 29303 raft_consensus.cc:493] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.294837 29303 raft_consensus.cc:515] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.295395 29303 leader_election.cc:290] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:37.295420 28483 leader_election.cc:343] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 092a9aba9c6e48f5be9cf5d55ba93678
I20260618 13:59:37.295673 28799 tablet_bootstrap.cc:492] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:37.295780 28799 ts_tablet_manager.cc:1403] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.052s user 0.002s sys 0.001s
W20260618 13:59:37.296795 28617 leader_election.cc:343] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 3dd0eaa64271489d92aa380df877f257
I20260618 13:59:37.296950 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dd0eaa64271489d92aa380df877f257" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:37.297451 28799 raft_consensus.cc:359] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.297652 28799 raft_consensus.cc:385] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.297731 28799 raft_consensus.cc:740] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
W20260618 13:59:37.297816 28618 leader_election.cc:343] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.297923 28618 leader_election.cc:304] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.298077 28799 consensus_queue.cc:260] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:37.298327 28484 leader_election.cc:343] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 092a9aba9c6e48f5be9cf5d55ba93678
I20260618 13:59:37.298465 28484 leader_election.cc:304] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.298688 29749 raft_consensus.cc:2749] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:37.298744 28799 ts_tablet_manager.cc:1434] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:37.298914 29303 raft_consensus.cc:2749] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:37.286259 29745 rpcz_store.cc:276] 0618 13:59:32.943398 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.943442 (+ 44us) service_pool.cc:224] Handling call
0618 13:59:37.267583 (+4324141us) ts_tablet_manager.cc:1151] Deleting tablet d7a97176f72349cda221baaea61d18ad
0618 13:59:37.267592 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.282400 (+ 14808us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.285031 (+ 2631us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.286149 (+ 1118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10053,"tablet-delete.queue_time_us":4324024}
I20260618 13:59:37.303464 29745 tablet_replica.cc:333] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:37.303444 28797 tablet_bootstrap.cc:492] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:37.303718 28797 ts_tablet_manager.cc:1403] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.055s user 0.006s sys 0.003s
I20260618 13:59:37.303769 29745 raft_consensus.cc:2243] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.304025 29745 raft_consensus.cc:2272] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.304428 28797 raft_consensus.cc:359] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.304603 28797 raft_consensus.cc:385] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.304661 28797 raft_consensus.cc:740] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.304847 28797 consensus_queue.cc:260] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.304963 28799 tablet_bootstrap.cc:492] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:37.305325 28797 ts_tablet_manager.cc:1434] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:37.305577 28797 tablet_bootstrap.cc:492] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:37.306663 28799 tablet_bootstrap.cc:654] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.307200 28797 tablet_bootstrap.cc:654] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.312146 30151 raft_consensus.cc:493] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.312270 30151 raft_consensus.cc:515] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.312841 30151 leader_election.cc:290] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:37.281375 29566 rpcz_store.cc:276] 0618 13:59:32.097855 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.097906 (+ 51us) service_pool.cc:224] Handling call
0618 13:59:37.215649 (+5117743us) ts_tablet_manager.cc:1151] Deleting tablet b51d0851efb34eaa9e9e4be6b292ef62
0618 13:59:37.215660 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.221246 (+ 5586us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.280034 (+ 58788us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.281250 (+ 1216us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5117586}
I20260618 13:59:37.313238 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1663f9d92e54f1caf3c7137412e495c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:37.313410 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 82) took 5214 ms. Trace:
I20260618 13:59:37.313503 29566 rpcz_store.cc:276] 0618 13:59:32.099039 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.099091 (+ 52us) service_pool.cc:224] Handling call
0618 13:59:37.313292 (+5214201us) ts_tablet_manager.cc:1151] Deleting tablet d4667fe48f0f4eff9c6c8baabdab58d4
0618 13:59:37.313301 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.313394 (+ 93us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5214038}
I20260618 13:59:37.313747 29566 tablet_replica.cc:333] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:37.314018 29566 raft_consensus.cc:2243] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:37.314011 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1663f9d92e54f1caf3c7137412e495c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:37.314455 28346 leader_election.cc:343] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.314560 29566 raft_consensus.cc:2272] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:37.314874 28346 leader_election.cc:343] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.315019 28346 leader_election.cc:304] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.315196 29566 ts_tablet_manager.cc:1916] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.315328 30151 raft_consensus.cc:2749] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:37.315584 29749 raft_consensus.cc:493] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6fc808af0e884ed8836561665d8e0e7b)
I20260618 13:59:37.315754 29749 raft_consensus.cc:515] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.316511 29749 leader_election.cc:290] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:37.317173 28483 leader_election.cc:343] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 41107eb1d9164a9c98060003b4027412
W20260618 13:59:37.317282 28484 leader_election.cc:343] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 41107eb1d9164a9c98060003b4027412
I20260618 13:59:37.317394 28484 leader_election.cc:304] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.267584 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 7aa9f65d8a5c46fdb745ee17af946dce (table default.Table_c2afd6643b7b49b7992384135658ddfc [id=29178416a03447608c28e809919d9461]) successfully deleted
W20260618 13:59:37.318688 28988 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 51101.5us
I20260618 13:59:37.318921 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet d7a97176f72349cda221baaea61d18ad (table default.Table_b0e3e2341f764392af42b949d03dd1f6 [id=27b6a19a28f24f2b8a758bbff06312d0]) successfully deleted
I20260618 13:59:37.268291 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet dd2385e6752d49429422fc854539e9c6 (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]) successfully deleted
W20260618 13:59:37.319110 28989 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 50858.6us
I20260618 13:59:37.319419 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet b51d0851efb34eaa9e9e4be6b292ef62 (table default.Table_7d317fa28e3142a19d15c8360151c357 [id=bba3ab8821ca48a5aa1104094d78892a]) successfully deleted
W20260618 13:59:37.319633 28989 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d4667fe48f0f4eff9c6c8baabdab58d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d4667fe48f0f4eff9c6c8baabdab58d4 already in progress: creating tablet
I20260618 13:59:37.320712 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:37.321596 30151 raft_consensus.cc:493] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.321717 30151 raft_consensus.cc:515] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.322302 30151 leader_election.cc:290] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:37.304569 29745 ts_tablet_manager.cc:1916] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.323136 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebebb79db644535b54a48e4dee89d59" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:37.323501 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:37.323822 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebebb79db644535b54a48e4dee89d59" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:37.324347 28346 leader_election.cc:343] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:37.324517 28346 leader_election.cc:343] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.324586 28346 leader_election.cc:304] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.324642 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:37.324988 30151 raft_consensus.cc:2749] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:37.324892 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:37.325183 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:37.325489 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:37.332988 28797 tablet_bootstrap.cc:492] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:37.333174 28797 ts_tablet_manager.cc:1403] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.000s
I20260618 13:59:37.334025 28797 raft_consensus.cc:359] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.334360 28797 raft_consensus.cc:385] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.334508 28797 raft_consensus.cc:740] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.334769 28797 consensus_queue.cc:260] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.334821 30027 consensus_queue.cc:1048] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:37.335302 28797 ts_tablet_manager.cc:1434] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 13:59:37.335768 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_7f787ab851be496dae97cba8835db19b [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 1 got 0
I20260618 13:59:37.336712 28797 tablet_bootstrap.cc:492] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:37.337930 30152 consensus_queue.cc:1048] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:37.338435 28797 tablet_bootstrap.cc:654] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.340731 29746 ts_tablet_manager.cc:1929] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.340984 29746 log.cc:1199] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/233040a16b0d4b61b977ca57c1afec2d
I20260618 13:59:37.341485 29746 ts_tablet_manager.cc:1950] T 233040a16b0d4b61b977ca57c1afec2d P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:37.342584 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 93) took 2767 ms. Trace:
I20260618 13:59:37.347990 29566 ts_tablet_manager.cc:1929] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260618 13:59:37.348500 28989 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ab100ebbe61a4321b9e76c730e5d46ed consensus state has no leader
I20260618 13:59:37.355013 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:37.355732 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:37.356060 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 72c04f55890648f0a8184f3783243f7c (table default.Table_773be314d7e741fcb4fb6eb7d954802e [id=0d83fc68255c458587390c58b12e00d7]). Expected version 2 got 0
I20260618 13:59:37.356418 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:37.356771 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a]). Expected version 1 got 0
I20260618 13:59:37.357211 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:37.357555 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:37.358067 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:37.352618 30152 raft_consensus.cc:493] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.358479 28799 tablet_bootstrap.cc:492] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:37.358459 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:37.358597 28799 ts_tablet_manager.cc:1403] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.054s user 0.008s sys 0.011s
I20260618 13:59:37.351178 29745 ts_tablet_manager.cc:1929] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.358809 29745 log.cc:1199] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/b0285c75c632457383e81805ab036aaf
I20260618 13:59:37.358785 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:37.359088 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:37.359303 29745 ts_tablet_manager.cc:1950] T b0285c75c632457383e81805ab036aaf P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:37.359400 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:37.359421 28799 raft_consensus.cc:359] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.359579 28799 raft_consensus.cc:385] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.359642 28799 raft_consensus.cc:740] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.359781 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:37.359844 28799 consensus_queue.cc:260] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.360283 28799 ts_tablet_manager.cc:1434] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:37.360538 28799 tablet_bootstrap.cc:492] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:37.360642 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 70) took 4424 ms. Trace:
I20260618 13:59:37.360741 29745 rpcz_store.cc:276] 0618 13:59:32.936442 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.936485 (+ 43us) service_pool.cc:224] Handling call
0618 13:59:37.303416 (+4366931us) ts_tablet_manager.cc:1151] Deleting tablet b0285c75c632457383e81805ab036aaf
0618 13:59:37.303426 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.333422 (+ 29996us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.351132 (+ 17710us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.360623 (+ 9491us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4359461}
I20260618 13:59:37.361109 29745 tablet_replica.cc:333] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:37.361387 29745 raft_consensus.cc:2243] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.361586 29745 raft_consensus.cc:2272] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.362134 29745 ts_tablet_manager.cc:1916] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.358378 30152 raft_consensus.cc:515] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.363199 30152 leader_election.cc:290] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:37.367758 30151 raft_consensus.cc:493] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.367882 30151 raft_consensus.cc:515] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:37.368353 28346 consensus_peers.cc:597] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:37.368422 30151 leader_election.cc:290] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:37.369225 28799 tablet_bootstrap.cc:654] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.362679 28549 raft_consensus.cc:1275] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2 [term 2 FOLLOWER]: Refusing update from remote peer 6fc808af0e884ed8836561665d8e0e7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260618 13:59:37.364912 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "650ed7ee8b134bc4b9f95028c54f99be" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:37.375751 28798 tablet_bootstrap.cc:492] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:37.375890 28798 ts_tablet_manager.cc:1403] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.094s user 0.005s sys 0.013s
I20260618 13:59:37.375738 29746 rpcz_store.cc:276] 0618 13:59:34.575243 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.575319 (+ 76us) service_pool.cc:224] Handling call
0618 13:59:37.268457 (+2693138us) ts_tablet_manager.cc:1151] Deleting tablet 233040a16b0d4b61b977ca57c1afec2d
0618 13:59:37.268467 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.310625 (+ 42158us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.340663 (+ 30038us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.342559 (+ 1896us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2692907}
I20260618 13:59:37.376601 28798 raft_consensus.cc:359] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.376464 29746 tablet_replica.cc:333] T 3a0039a463ec4a9c82a21a3a69b565fc P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:37.376778 28798 raft_consensus.cc:385] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.376827 28798 raft_consensus.cc:740] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.376920 29746 raft_consensus.cc:2243] T 3a0039a463ec4a9c82a21a3a69b565fc P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.377038 29746 raft_consensus.cc:2272] T 3a0039a463ec4a9c82a21a3a69b565fc P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.376991 28798 consensus_queue.cc:260] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.369037 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f7b66652f464893ad812d27e883a9a4" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:37.378393 28346 leader_election.cc:343] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:37.378655 28346 leader_election.cc:343] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:37.371832 28484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.3:35503, user_credentials={real_user=slave}} blocked reactor thread for 54615.5us
I20260618 13:59:37.373737 29749 raft_consensus.cc:2749] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:37.374321 30165 raft_consensus.cc:493] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.381451 30165 raft_consensus.cc:515] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.382126 30165 leader_election.cc:290] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:37.383138 28483 leader_election.cc:343] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: d1baf86497d442dbbbddbc3498bebbee
I20260618 13:59:37.383821 30152 consensus_queue.cc:1048] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:37.385514 28484 leader_election.cc:343] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: d1baf86497d442dbbbddbc3498bebbee
I20260618 13:59:37.385614 28484 leader_election.cc:304] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.385834 30165 raft_consensus.cc:2749] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:37.386502 28798 ts_tablet_manager.cc:1434] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260618 13:59:37.386848 28798 tablet_bootstrap.cc:492] T c3d523419ff244f085d6eac1a3231973 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:37.388526 28798 tablet_bootstrap.cc:654] T c3d523419ff244f085d6eac1a3231973 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:37.390854 28346 consensus_peers.cc:597] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b -> Peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Couldn't send request to peer d6a169b16ced46ca9cfbc631709bb257. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aef7c1f111634c98af046e73fa9ffdeb. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:37.365777 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "650ed7ee8b134bc4b9f95028c54f99be" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:37.394738 29566 log.cc:1199] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/d7a97176f72349cda221baaea61d18ad
I20260618 13:59:37.394795 28682 raft_consensus.cc:2468] T 650ed7ee8b134bc4b9f95028c54f99be P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:37.369462 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f7b66652f464893ad812d27e883a9a4" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:37.395118 29566 ts_tablet_manager.cc:1950] T d7a97176f72349cda221baaea61d18ad P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:37.395238 28346 leader_election.cc:304] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.376750 28730 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
W20260618 13:59:37.385300 28618 consensus_peers.cc:597] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:37.387893 28797 tablet_bootstrap.cc:492] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:37.391906 29644 raft_consensus.cc:493] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.395651 29644 raft_consensus.cc:515] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:37.395879 28346 leader_election.cc:343] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.395987 28346 leader_election.cc:304] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.396171 29644 leader_election.cc:290] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.396279 30152 raft_consensus.cc:2804] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:37.396353 30152 raft_consensus.cc:493] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:37.396415 30152 raft_consensus.cc:3060] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.396652 28797 ts_tablet_manager.cc:1403] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.060s user 0.003s sys 0.000s
I20260618 13:59:37.396792 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d8dd2c388e43a0be380aa953ec2c82" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:37.396938 28403 raft_consensus.cc:2468] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:37.397297 28617 leader_election.cc:304] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:37.397433 28797 raft_consensus.cc:359] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.397540 29400 raft_consensus.cc:2804] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:37.397570 28797 raft_consensus.cc:385] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.397598 29400 raft_consensus.cc:493] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:37.397614 28797 raft_consensus.cc:740] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.397650 29400 raft_consensus.cc:3060] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.397907 28797 consensus_queue.cc:260] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.398346 28797 ts_tablet_manager.cc:1434] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:37.398607 28797 tablet_bootstrap.cc:492] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:37.398891 30152 raft_consensus.cc:515] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.399411 30152 leader_election.cc:290] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:37.362519 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet b0285c75c632457383e81805ab036aaf (table default.Table_9cd59394f11d4937887809241a69bba3 [id=75fcc464e22646f7960501f112b3f887]) successfully deleted
I20260618 13:59:37.376235 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 233040a16b0d4b61b977ca57c1afec2d (table default.Table_7e5df99a84c84d66b548875d6d934656 [id=2332167cba774a0795c11d97505049c9]) successfully deleted
I20260618 13:59:37.400044 29400 raft_consensus.cc:515] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.400158 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:37.400384 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:37.400524 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:37.400590 29400 leader_election.cc:290] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.400863 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_c70d6c5b0dc94b33b4c5772edd2b9dca [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 2 got 0
I20260618 13:59:37.401147 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:37.401295 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d8dd2c388e43a0be380aa953ec2c82" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:37.401420 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:37.401468 28404 raft_consensus.cc:3060] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.401705 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:37.402052 29023 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:37.402498 29019 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_dbc6162c06f74e71b792bd1f7b2388fe [id=49576dabdecf41aca1b15ff3fc0c113c] alter complete (version 6)
I20260618 13:59:37.402573 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d8dd2c388e43a0be380aa953ec2c82" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:37.402897 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d8dd2c388e43a0be380aa953ec2c82" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:37.403069 28618 leader_election.cc:343] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.403930 28404 raft_consensus.cc:2468] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:37.403995 30152 raft_consensus.cc:2749] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:37.404516 28617 leader_election.cc:304] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.404721 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "650ed7ee8b134bc4b9f95028c54f99be" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:37.404997 29714 raft_consensus.cc:2804] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:37.402520 29026 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet cea4bc13eabc4dd98b4f96723498014c
I20260618 13:59:37.405128 29714 raft_consensus.cc:697] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
W20260618 13:59:37.405165 28346 leader_election.cc:343] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.405339 29714 consensus_queue.cc:237] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.405632 29026 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 100) took 3522 ms. Trace:
I20260618 13:59:37.405735 29026 rpcz_store.cc:276] 0618 13:59:33.883130 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.426185 (+1543055us) service_pool.cc:224] Handling call
0618 13:59:35.433791 (+ 7606us) spinlock_profiling.cc:243] Waited 7.37 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb46856b6 00007fcbab0e2486 00007fcbab0e21b5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:35.434305 (+ 514us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:35.434465 (+ 160us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:35.434532 (+ 67us) catalog_manager.cc:2849] Locking tablets
0618 13:59:35.434657 (+ 125us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:35.464838 (+ 30181us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:35.560031 (+ 95193us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:37.402452 (+1842421us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:37.405573 (+ 3121us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:35.436209 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:35.436305 (+ 96us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:35.436310 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:35.436312 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:35.436466 (+ 154us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:35.436476 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:35.436480 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:35.436565 (+ 85us) tablet.cc:701] Row locks acquired
0618 13:59:35.436567 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:35.436599 (+ 32us) write_op.cc:270] Start()
0618 13:59:35.436627 (+ 28us) write_op.cc:276] Timestamp: P: 1781791175436563 usec, L: 0
0618 13:59:35.436631 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:35.436788 (+ 157us) log.cc:844] Serialized 793 byte log entry
0618 13:59:35.463828 (+ 27040us) op_driver.cc:464] REPLICATION: finished
0618 13:59:35.464217 (+ 389us) write_op.cc:301] APPLY: starting
0618 13:59:35.464234 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:35.464291 (+ 57us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:35.464370 (+ 79us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:35.464446 (+ 76us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:35.464479 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:35.464487 (+ 8us) write_op.cc:312] APPLY: finished
0618 13:59:35.464567 (+ 80us) log.cc:844] Serialized 29 byte log entry
0618 13:59:35.464630 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.464649 (+ 19us) write_op.cc:454] Released schema lock
0618 13:59:35.464655 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:35.464728 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.464732 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":16953728,"child_traces":[["op",{"apply.queue_time_us":315,"apply.run_cpu_time_us":520,"apply.run_wall_time_us":520,"num_ops":2,"prepare.queue_time_us":42,"prepare.run_cpu_time_us":641,"prepare.run_wall_time_us":641,"replication_time_us":27161}]]}
W20260618 13:59:37.406752 29026 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.3:42065 (request call id 145) took 1034 ms (client timeout 1000 ms). Trace:
W20260618 13:59:37.406863 29026 rpcz_store.cc:269] 0618 13:59:36.372510 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:37.406465 (+1033955us) service_pool.cc:211] Skipping call since client already timed out
0618 13:59:37.406697 (+ 232us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260618 13:59:37.407353 28618 leader_election.cc:343] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.408267 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "650ed7ee8b134bc4b9f95028c54f99be" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:37.408428 28680 raft_consensus.cc:3060] T 650ed7ee8b134bc4b9f95028c54f99be P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.410892 28680 raft_consensus.cc:2468] T 650ed7ee8b134bc4b9f95028c54f99be P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:37.411283 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet cea4bc13eabc4dd98b4f96723498014c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:37.411490 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet cea4bc13eabc4dd98b4f96723498014c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:37.412125 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet cea4bc13eabc4dd98b4f96723498014c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:37.412416 28346 leader_election.cc:304] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.412818 30152 raft_consensus.cc:2804] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:37.412972 30152 raft_consensus.cc:697] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:37.413288 30152 consensus_queue.cc:237] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.424536 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 83) took 5323 ms. Trace:
I20260618 13:59:37.424924 29566 rpcz_store.cc:276] 0618 13:59:32.100660 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.100707 (+ 47us) service_pool.cc:224] Handling call
0618 13:59:37.313708 (+5213001us) ts_tablet_manager.cc:1151] Deleting tablet d7a97176f72349cda221baaea61d18ad
0618 13:59:37.313715 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.344724 (+ 31009us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.347950 (+ 3226us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.424511 (+ 76561us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5212856}
I20260618 13:59:37.425657 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet d7a97176f72349cda221baaea61d18ad (table default.Table_b0e3e2341f764392af42b949d03dd1f6 [id=27b6a19a28f24f2b8a758bbff06312d0]) successfully deleted
I20260618 13:59:37.426059 29566 tablet_replica.cc:333] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:37.426378 29566 raft_consensus.cc:2243] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.426637 29566 raft_consensus.cc:2272] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.427472 29566 ts_tablet_manager.cc:1916] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.434192 29400 consensus_queue.cc:1048] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:37.435365 28799 tablet_bootstrap.cc:492] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:37.435510 28799 ts_tablet_manager.cc:1403] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.075s user 0.003s sys 0.014s
I20260618 13:59:37.435530 29400 raft_consensus.cc:493] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.435645 29400 raft_consensus.cc:515] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.436254 29400 leader_election.cc:290] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:37.436205 28799 raft_consensus.cc:359] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.436352 28799 raft_consensus.cc:385] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.436412 28799 raft_consensus.cc:740] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.436553 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f7359c4a144ca89fe3e51d1365a425" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:37.436592 28799 consensus_queue.cc:260] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.436712 28404 raft_consensus.cc:2468] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:37.437059 28799 ts_tablet_manager.cc:1434] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260618 13:59:37.437283 28799 tablet_bootstrap.cc:492] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:37.437480 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f7359c4a144ca89fe3e51d1365a425" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:37.438230 28618 leader_election.cc:343] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.439023 28617 leader_election.cc:304] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.439383 29400 raft_consensus.cc:2804] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:37.439453 29400 raft_consensus.cc:493] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:37.439512 29400 raft_consensus.cc:3060] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.439996 28799 tablet_bootstrap.cc:654] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.442795 29400 raft_consensus.cc:515] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.443596 29400 leader_election.cc:290] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:37.445736 29644 raft_consensus.cc:493] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.445856 29644 raft_consensus.cc:515] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.446326 29644 leader_election.cc:290] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:37.447463 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f7359c4a144ca89fe3e51d1365a425" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:37.447603 28404 raft_consensus.cc:3060] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.448807 28401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15483938a06e459789f87c6bb456bd15" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:37.448941 28401 raft_consensus.cc:2468] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:37.450012 28404 raft_consensus.cc:2468] T f9f7359c4a144ca89fe3e51d1365a425 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:37.450829 28617 leader_election.cc:304] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:37.451211 28617 leader_election.cc:304] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:37.451483 29303 raft_consensus.cc:2804] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:37.451584 29303 raft_consensus.cc:697] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:37.451777 29303 consensus_queue.cc:237] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.451239 29400 raft_consensus.cc:2804] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:37.453027 29400 raft_consensus.cc:493] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:37.453105 29400 raft_consensus.cc:3060] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.453761 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f7359c4a144ca89fe3e51d1365a425" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:37.454084 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15483938a06e459789f87c6bb456bd15" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:37.455461 28618 leader_election.cc:343] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:37.455667 28618 leader_election.cc:343] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.456066 29644 raft_consensus.cc:493] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.456169 29644 raft_consensus.cc:515] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.456724 29644 leader_election.cc:290] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.461270 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9024952ab8094fcda1329562bbde8ef5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:37.461439 28403 raft_consensus.cc:2468] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:37.462916 28617 leader_election.cc:304] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:37.463132 30027 raft_consensus.cc:2804] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:37.463213 30027 raft_consensus.cc:493] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:37.463268 30027 raft_consensus.cc:3060] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.465627 30027 raft_consensus.cc:515] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.466069 30027 leader_election.cc:290] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:37.466703 29017 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3a783590159f4a80964fbbcb4a24eef9 consensus state has no leader
I20260618 13:59:37.467242 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9024952ab8094fcda1329562bbde8ef5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:37.467388 28404 raft_consensus.cc:3060] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.467830 29400 raft_consensus.cc:515] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.468490 29400 leader_election.cc:290] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:37.468842 29746 ts_tablet_manager.cc:1916] T 3a0039a463ec4a9c82a21a3a69b565fc P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.472298 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9024952ab8094fcda1329562bbde8ef5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:37.472514 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15483938a06e459789f87c6bb456bd15" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:37.473008 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9024952ab8094fcda1329562bbde8ef5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:37.473937 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15483938a06e459789f87c6bb456bd15" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:37.474102 28403 raft_consensus.cc:3060] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.476665 28403 raft_consensus.cc:2468] T 15483938a06e459789f87c6bb456bd15 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
W20260618 13:59:37.477226 28618 leader_election.cc:343] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:37.477552 28618 leader_election.cc:343] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.478137 28617 leader_election.cc:304] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.478566 30027 raft_consensus.cc:2804] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:37.478804 30027 raft_consensus.cc:697] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:37.479032 30027 consensus_queue.cc:237] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.479342 28797 tablet_bootstrap.cc:654] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:37.481271 28618 leader_election.cc:343] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:37.485035 29024 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d15b1b603f024f3295eb07c98991076d consensus state has no leader
I20260618 13:59:37.485284 29018 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet c01c8463055e4526b3c75190066f30b6
I20260618 13:59:37.485751 29018 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 101) took 3354 ms. Trace:
I20260618 13:59:37.485873 29018 rpcz_store.cc:276] 0618 13:59:34.131039 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.427905 (+1296866us) service_pool.cc:224] Handling call
0618 13:59:35.440510 (+ 12605us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb46856b6 00007fcbab0e2486 00007fcbab0e21b5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:35.561083 (+120573us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:35.561228 (+ 145us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:35.561278 (+ 50us) catalog_manager.cc:2849] Locking tablets
0618 13:59:35.704171 (+142893us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:35.719984 (+ 15813us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:35.720071 (+ 87us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:37.485229 (+1765158us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:37.485718 (+ 489us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:35.704887 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:35.704987 (+ 100us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:35.704993 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:35.704995 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:35.705098 (+ 103us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:35.705106 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:35.705112 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:35.705203 (+ 91us) tablet.cc:701] Row locks acquired
0618 13:59:35.705207 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:35.705236 (+ 29us) write_op.cc:270] Start()
0618 13:59:35.705262 (+ 26us) write_op.cc:276] Timestamp: P: 1781791175705173 usec, L: 0
0618 13:59:35.705265 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:35.705422 (+ 157us) log.cc:844] Serialized 728 byte log entry
0618 13:59:35.718989 (+ 13567us) op_driver.cc:464] REPLICATION: finished
0618 13:59:35.719360 (+ 371us) write_op.cc:301] APPLY: starting
0618 13:59:35.719387 (+ 27us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:35.719449 (+ 62us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:35.719475 (+ 26us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:35.719537 (+ 62us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:35.719567 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:35.719574 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:35.719643 (+ 69us) log.cc:844] Serialized 29 byte log entry
0618 13:59:35.719875 (+ 232us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.719895 (+ 20us) write_op.cc:454] Released schema lock
0618 13:59:35.719903 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:35.722776 (+ 2873us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.722782 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":27955456,"child_traces":[["op",{"apply.queue_time_us":316,"apply.run_cpu_time_us":453,"apply.run_wall_time_us":3405,"num_ops":2,"prepare.queue_time_us":51,"prepare.run_cpu_time_us":597,"prepare.run_wall_time_us":597,"replication_time_us":13691}]]}
I20260618 13:59:37.488497 28404 raft_consensus.cc:2468] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:37.488894 28617 leader_election.cc:304] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.489130 30059 raft_consensus.cc:2804] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:37.489221 30059 raft_consensus.cc:697] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:37.489440 30059 consensus_queue.cc:237] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.493558 28799 tablet_bootstrap.cc:492] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:37.493677 28799 ts_tablet_manager.cc:1403] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.056s user 0.009s sys 0.017s
I20260618 13:59:37.493958 30152 raft_consensus.cc:493] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.494066 30152 raft_consensus.cc:515] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.494370 28799 raft_consensus.cc:359] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.494539 28799 raft_consensus.cc:385] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.494587 28799 raft_consensus.cc:740] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.494647 30152 leader_election.cc:290] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:37.494800 28799 consensus_queue.cc:260] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.495204 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9f9649587b4c31951204b8bebe7ed8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:37.495357 28799 ts_tablet_manager.cc:1434] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:37.495617 28799 tablet_bootstrap.cc:492] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:37.495666 28346 leader_election.cc:343] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.495980 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9f9649587b4c31951204b8bebe7ed8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:37.496959 28346 leader_election.cc:343] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.497045 28346 leader_election.cc:304] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.497236 28799 tablet_bootstrap.cc:654] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.498131 30152 raft_consensus.cc:2749] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:37.498951 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet c01c8463055e4526b3c75190066f30b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:37.498971 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet c01c8463055e4526b3c75190066f30b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:37.499486 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet c01c8463055e4526b3c75190066f30b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:37.501310 30152 raft_consensus.cc:493] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.501428 30152 raft_consensus.cc:515] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.501976 30152 leader_election.cc:290] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:37.502667 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bcff62290c43c9be7cc496fbff6142" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:37.503322 28346 leader_election.cc:343] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.503649 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bcff62290c43c9be7cc496fbff6142" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:37.504281 28346 leader_election.cc:343] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.504397 28346 leader_election.cc:304] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.504784 30152 raft_consensus.cc:2749] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:37.512174 28799 tablet_bootstrap.cc:492] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:37.512353 28799 ts_tablet_manager.cc:1403] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.003s
I20260618 13:59:37.513051 28799 raft_consensus.cc:359] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.513247 28799 raft_consensus.cc:385] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.513346 28799 raft_consensus.cc:740] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.513485 29025 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 6c4a31a4f202473d97b17b5b5dfdb615
I20260618 13:59:37.513613 28799 consensus_queue.cc:260] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.513695 29746 ts_tablet_manager.cc:1929] T 3a0039a463ec4a9c82a21a3a69b565fc P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:37.513784 29746 log.cc:1199] T 3a0039a463ec4a9c82a21a3a69b565fc P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/3a0039a463ec4a9c82a21a3a69b565fc
I20260618 13:59:37.514081 29746 ts_tablet_manager.cc:1950] T 3a0039a463ec4a9c82a21a3a69b565fc P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:37.514050 29025 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 102) took 3327 ms. Trace:
I20260618 13:59:37.514214 28799 ts_tablet_manager.cc:1434] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:37.514487 28799 tablet_bootstrap.cc:492] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:37.514144 29025 rpcz_store.cc:276] 0618 13:59:34.186961 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.559463 (+1372502us) service_pool.cc:224] Handling call
0618 13:59:35.560436 (+ 973us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:35.560581 (+ 145us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:35.560638 (+ 57us) catalog_manager.cc:2849] Locking tablets
0618 13:59:35.676773 (+116135us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:35.690609 (+ 13836us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:35.690697 (+ 88us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:37.513436 (+1822739us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:37.513995 (+ 559us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:35.677856 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:35.677989 (+ 133us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:35.677997 (+ 8us) write_op.cc:435] Acquired schema lock
0618 13:59:35.678001 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:35.678118 (+ 117us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:35.678131 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:35.678138 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:35.678277 (+ 139us) tablet.cc:701] Row locks acquired
0618 13:59:35.678280 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:35.678323 (+ 43us) write_op.cc:270] Start()
0618 13:59:35.678355 (+ 32us) write_op.cc:276] Timestamp: P: 1781791175678259 usec, L: 0
0618 13:59:35.678360 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:35.678566 (+ 206us) log.cc:844] Serialized 728 byte log entry
0618 13:59:35.687279 (+ 8713us) op_driver.cc:464] REPLICATION: finished
0618 13:59:35.690124 (+ 2845us) write_op.cc:301] APPLY: starting
0618 13:59:35.690140 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:35.690181 (+ 41us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:35.690213 (+ 32us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:35.690272 (+ 59us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:35.690302 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:35.690311 (+ 9us) write_op.cc:312] APPLY: finished
0618 13:59:35.690376 (+ 65us) log.cc:844] Serialized 29 byte log entry
0618 13:59:35.690458 (+ 82us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.690474 (+ 16us) write_op.cc:454] Released schema lock
0618 13:59:35.690480 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:35.690603 (+ 123us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.690608 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2790,"apply.run_cpu_time_us":437,"apply.run_wall_time_us":473,"num_ops":2,"prepare.queue_time_us":314,"prepare.run_cpu_time_us":828,"prepare.run_wall_time_us":827,"raft.queue_time_us":812,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":5506,"replication_time_us":8890}]]}
I20260618 13:59:37.514772 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 94) took 2900 ms. Trace:
I20260618 13:59:37.514842 29746 rpcz_store.cc:276] 0618 13:59:34.614498 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.614555 (+ 57us) service_pool.cc:224] Handling call
0618 13:59:37.376418 (+2761863us) ts_tablet_manager.cc:1151] Deleting tablet 3a0039a463ec4a9c82a21a3a69b565fc
0618 13:59:37.376427 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.491399 (+114972us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.513641 (+ 22242us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.514755 (+ 1114us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2761629}
I20260618 13:59:37.515204 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 95) took 2894 ms. Trace:
I20260618 13:59:37.515280 29746 rpcz_store.cc:276] 0618 13:59:34.620749 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.620845 (+ 96us) service_pool.cc:224] Handling call
0618 13:59:37.515110 (+2894265us) ts_tablet_manager.cc:1151] Deleting tablet 2188ebf1727240cd886ba7662f8fcf43
0618 13:59:37.515121 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.515189 (+ 68us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2894044}
I20260618 13:59:37.515592 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 96) took 2894 ms. Trace:
I20260618 13:59:37.515656 29746 rpcz_store.cc:276] 0618 13:59:34.621378 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.621433 (+ 55us) service_pool.cc:224] Handling call
0618 13:59:37.515486 (+2894053us) ts_tablet_manager.cc:1151] Deleting tablet d4667fe48f0f4eff9c6c8baabdab58d4
0618 13:59:37.515505 (+ 19us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.515578 (+ 73us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2893889}
I20260618 13:59:37.515862 29746 tablet_replica.cc:333] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:37.516065 28799 tablet_bootstrap.cc:654] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.519699 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 6c4a31a4f202473d97b17b5b5dfdb615 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:37.520020 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 3a0039a463ec4a9c82a21a3a69b565fc (table default.Table_8c1fdd54393c4ad383e72a07837ead2a [id=6a7e8e3becb64fd5bc583ee08eaa4437]) successfully deleted
W20260618 13:59:37.520262 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 2188ebf1727240cd886ba7662f8fcf43 because the tablet was not found. No further retry: Not found: Tablet not found: 2188ebf1727240cd886ba7662f8fcf43
W20260618 13:59:37.520480 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d4667fe48f0f4eff9c6c8baabdab58d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d4667fe48f0f4eff9c6c8baabdab58d4 already in progress: creating tablet
I20260618 13:59:37.521013 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 6c4a31a4f202473d97b17b5b5dfdb615 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:37.521203 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet 6c4a31a4f202473d97b17b5b5dfdb615 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:37.521526 29017 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 103) took 3248 ms. Trace:
I20260618 13:59:37.521653 29017 rpcz_store.cc:276] 0618 13:59:34.272577 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.571395 (+1298818us) service_pool.cc:224] Handling call
0618 13:59:35.581425 (+ 10030us) spinlock_profiling.cc:243] Waited 9.75 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb4686956 00007fcbab0e4146 00007fcbab0e3e75 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:35.581766 (+ 341us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:35.581996 (+ 230us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:35.620934 (+ 38938us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:35.621034 (+ 100us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:35.626916 (+ 5882us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:37.521481 (+1894565us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:35.621641 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:35.621756 (+ 115us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:35.621761 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:35.621764 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:35.621855 (+ 91us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:35.621868 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:35.621887 (+ 19us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:35.621947 (+ 60us) tablet.cc:701] Row locks acquired
0618 13:59:35.621949 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:35.621985 (+ 36us) write_op.cc:270] Start()
0618 13:59:35.622016 (+ 31us) write_op.cc:276] Timestamp: P: 1781791175621928 usec, L: 0
0618 13:59:35.622019 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:35.622189 (+ 170us) log.cc:844] Serialized 405 byte log entry
0618 13:59:35.626033 (+ 3844us) op_driver.cc:464] REPLICATION: finished
0618 13:59:35.626357 (+ 324us) write_op.cc:301] APPLY: starting
0618 13:59:35.626372 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:35.626438 (+ 66us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:35.626464 (+ 26us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:35.626514 (+ 50us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:35.626549 (+ 35us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:35.626556 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:35.626625 (+ 69us) log.cc:844] Serialized 23 byte log entry
0618 13:59:35.626685 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.626702 (+ 17us) write_op.cc:454] Released schema lock
0618 13:59:35.626708 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:35.626777 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.626781 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":22427520,"child_traces":[["op",{"apply.queue_time_us":267,"apply.run_cpu_time_us":415,"apply.run_wall_time_us":414,"num_ops":1,"prepare.queue_time_us":55,"prepare.run_cpu_time_us":651,"prepare.run_wall_time_us":652,"raft.queue_time_us":69,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":273,"replication_time_us":3981}]]}
I20260618 13:59:37.523365 29017 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_2a3bcaddd3034d67a5ea7c78f0ea9e7c" } new_table_name: "default.Table_b7f097c1abe1410a800ce5aa25771740" modify_external_catalogs: true
I20260618 13:59:37.521543 29024 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 104) took 3131 ms. Trace:
I20260618 13:59:37.528432 29024 rpcz_store.cc:276] 0618 13:59:34.390177 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.695071 (+1304894us) service_pool.cc:224] Handling call
0618 13:59:35.695436 (+ 365us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:35.695640 (+ 204us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:35.695798 (+ 158us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:35.695870 (+ 72us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:35.713201 (+ 17331us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:37.521504 (+1808303us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:35.700385 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:35.700518 (+ 133us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:35.700525 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:35.700529 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:35.700629 (+ 100us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:35.700643 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:35.700650 (+ 7us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:35.700712 (+ 62us) tablet.cc:701] Row locks acquired
0618 13:59:35.700716 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:35.700752 (+ 36us) write_op.cc:270] Start()
0618 13:59:35.700784 (+ 32us) write_op.cc:276] Timestamp: P: 1781791175700687 usec, L: 0
0618 13:59:35.700789 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:35.700971 (+ 182us) log.cc:844] Serialized 405 byte log entry
0618 13:59:35.712192 (+ 11221us) op_driver.cc:464] REPLICATION: finished
0618 13:59:35.712574 (+ 382us) write_op.cc:301] APPLY: starting
0618 13:59:35.712590 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:35.712633 (+ 43us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:35.712669 (+ 36us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:35.712715 (+ 46us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:35.712746 (+ 31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:35.712752 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:35.712824 (+ 72us) log.cc:844] Serialized 23 byte log entry
0618 13:59:35.713027 (+ 203us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.713041 (+ 14us) write_op.cc:454] Released schema lock
0618 13:59:35.713048 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:35.713114 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:35.713117 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":326,"apply.run_cpu_time_us":392,"apply.run_wall_time_us":535,"num_ops":1,"prepare.queue_time_us":4100,"prepare.run_cpu_time_us":703,"prepare.run_wall_time_us":703,"raft.queue_time_us":2662,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":238,"replication_time_us":11372}]]}
I20260618 13:59:37.529750 29024 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_e33476def9e34806972e63944bfc2998"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:37.530408 29024 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e33476def9e34806972e63944bfc2998 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:37.536047 30152 raft_consensus.cc:493] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.536177 30152 raft_consensus.cc:515] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.536722 30152 leader_election.cc:290] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:37.537564 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4667fe48f0f4eff9c6c8baabdab58d4" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:37.538028 28346 leader_election.cc:343] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.540623 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4667fe48f0f4eff9c6c8baabdab58d4" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:37.541298 28346 leader_election.cc:343] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.541383 28346 leader_election.cc:304] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.541788 30152 raft_consensus.cc:2749] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:37.541850 29566 ts_tablet_manager.cc:1929] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.541944 29566 log.cc:1199] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/516421cd843b44769efa2b555697c065
I20260618 13:59:37.542299 29566 ts_tablet_manager.cc:1950] T 516421cd843b44769efa2b555697c065 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:37.545907 29745 ts_tablet_manager.cc:1929] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.546094 29745 log.cc:1199] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/8fd9fa9675234db2abff694a517c0ea8
I20260618 13:59:37.546720 29745 ts_tablet_manager.cc:1950] T 8fd9fa9675234db2abff694a517c0ea8 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:37.547701 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 73) took 4604 ms. Trace:
I20260618 13:59:37.547863 29745 rpcz_store.cc:276] 0618 13:59:32.943138 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.943611 (+ 473us) service_pool.cc:224] Handling call
0618 13:59:37.361063 (+4417452us) ts_tablet_manager.cc:1151] Deleting tablet 8fd9fa9675234db2abff694a517c0ea8
0618 13:59:37.361077 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.371067 (+ 9990us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.545834 (+174767us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.547678 (+ 1844us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4411399}
I20260618 13:59:37.548657 29745 tablet_replica.cc:333] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:37.548604 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 8fd9fa9675234db2abff694a517c0ea8 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]) successfully deleted
I20260618 13:59:37.548902 29745 raft_consensus.cc:2243] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 2 LEADER]: Raft consensus shutting down.
I20260618 13:59:37.549216 29745 raft_consensus.cc:2272] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.550196 29745 ts_tablet_manager.cc:1916] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.556332 29745 ts_tablet_manager.cc:1929] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260618 13:59:37.556528 29745 log.cc:1199] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/aef7c1f111634c98af046e73fa9ffdeb
I20260618 13:59:37.556995 29745 ts_tablet_manager.cc:1950] T aef7c1f111634c98af046e73fa9ffdeb P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:37.557965 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 75) took 4614 ms. Trace:
I20260618 13:59:37.558137 29745 rpcz_store.cc:276] 0618 13:59:32.943320 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.943594 (+ 274us) service_pool.cc:224] Handling call
0618 13:59:37.548615 (+4605021us) ts_tablet_manager.cc:1151] Deleting tablet aef7c1f111634c98af046e73fa9ffdeb
0618 13:59:37.548624 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.553090 (+ 4466us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.556266 (+ 3176us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.557939 (+ 1673us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4592203}
I20260618 13:59:37.558837 29745 tablet_replica.cc:333] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:37.559152 29745 raft_consensus.cc:2243] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.559196 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet aef7c1f111634c98af046e73fa9ffdeb (table default.Table_191169025bff42cab93cf42503fa7220 [id=9c29cca38acc493aaec62cf3b0d93fcf]) successfully deleted
I20260618 13:59:37.559432 29745 raft_consensus.cc:2272] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.560142 29745 ts_tablet_manager.cc:1916] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.561530 29746 raft_consensus.cc:2243] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.561697 29746 raft_consensus.cc:2272] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.562186 29746 ts_tablet_manager.cc:1916] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.568154 29746 ts_tablet_manager.cc:1929] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.568337 29746 log.cc:1199] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/8b88ed7cf79c4929a8704f66691cbed4
I20260618 13:59:37.568837 29746 ts_tablet_manager.cc:1950] T 8b88ed7cf79c4929a8704f66691cbed4 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:37.569725 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 98) took 2946 ms. Trace:
I20260618 13:59:37.569901 29746 rpcz_store.cc:276] 0618 13:59:34.623004 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.623056 (+ 52us) service_pool.cc:224] Handling call
0618 13:59:37.515826 (+2892770us) ts_tablet_manager.cc:1151] Deleting tablet 8b88ed7cf79c4929a8704f66691cbed4
0618 13:59:37.515835 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.564950 (+ 49115us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.568098 (+ 3148us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.569701 (+ 1603us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45466,"tablet-delete.queue_time_us":2892607}
I20260618 13:59:37.570597 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 8b88ed7cf79c4929a8704f66691cbed4 (table default.Table_1333d91d141a46acac33460026215195 [id=232810f09f9e41269d67ccd55527ab8a]) successfully deleted
I20260618 13:59:37.571055 29746 tablet_replica.cc:333] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:37.571346 29746 raft_consensus.cc:2243] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.571760 29746 raft_consensus.cc:2272] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.576503 29745 ts_tablet_manager.cc:1929] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.577049 29746 ts_tablet_manager.cc:1916] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.583971 29745 log.cc:1199] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/751cd9bd961b4f56a588ed02a05f0212
I20260618 13:59:37.584839 29745 ts_tablet_manager.cc:1950] T 751cd9bd961b4f56a588ed02a05f0212 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:37.585623 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 74) took 4642 ms. Trace:
I20260618 13:59:37.585754 29745 rpcz_store.cc:276] 0618 13:59:32.943239 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.943573 (+ 334us) service_pool.cc:224] Handling call
0618 13:59:37.558762 (+4615189us) ts_tablet_manager.cc:1151] Deleting tablet 751cd9bd961b4f56a588ed02a05f0212
0618 13:59:37.558775 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.573806 (+ 15031us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.576460 (+ 2654us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.585604 (+ 9144us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4602193}
I20260618 13:59:37.586364 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 751cd9bd961b4f56a588ed02a05f0212 (table default.Table_74620afb05b24bf9912f3984efd632aa [id=ad88f16ed9e143cda1e892de63a18849]) successfully deleted
I20260618 13:59:37.586531 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 80) took 4629 ms. Trace:
I20260618 13:59:37.586721 29745 rpcz_store.cc:276] 0618 13:59:32.957185 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.957252 (+ 67us) service_pool.cc:224] Handling call
0618 13:59:37.586387 (+4629135us) ts_tablet_manager.cc:1151] Deleting tablet 233040a16b0d4b61b977ca57c1afec2d
0618 13:59:37.586402 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.586512 (+ 110us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4628959}
W20260618 13:59:37.587411 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 233040a16b0d4b61b977ca57c1afec2d because the tablet was not found. No further retry: Not found: Tablet not found: 233040a16b0d4b61b977ca57c1afec2d
I20260618 13:59:37.587492 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 81) took 4629 ms. Trace:
I20260618 13:59:37.587582 29745 rpcz_store.cc:276] 0618 13:59:32.957534 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.957653 (+ 119us) service_pool.cc:224] Handling call
0618 13:59:37.587396 (+4629743us) ts_tablet_manager.cc:1151] Deleting tablet c4cfca9e76e641a7bc0b58b9ee240f36
0618 13:59:37.587409 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.587476 (+ 67us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4629621}
I20260618 13:59:37.588119 29746 ts_tablet_manager.cc:1929] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:37.588475 29745 tablet_replica.cc:333] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
W20260618 13:59:37.588455 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet c4cfca9e76e641a7bc0b58b9ee240f36 because the tablet was not found. No further retry: Not found: Tablet not found: c4cfca9e76e641a7bc0b58b9ee240f36
I20260618 13:59:37.588620 29746 log.cc:1199] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/e025b64380314546af8a1ab90c817825
I20260618 13:59:37.588855 29745 raft_consensus.cc:2243] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.589195 29746 ts_tablet_manager.cc:1950] T e025b64380314546af8a1ab90c817825 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:37.589795 29745 raft_consensus.cc:2272] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.590544 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 99) took 2967 ms. Trace:
I20260618 13:59:37.591123 29745 ts_tablet_manager.cc:1916] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.591564 29746 rpcz_store.cc:276] 0618 13:59:34.623359 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.623404 (+ 45us) service_pool.cc:224] Handling call
0618 13:59:37.570997 (+2947593us) ts_tablet_manager.cc:1151] Deleting tablet e025b64380314546af8a1ab90c817825
0618 13:59:37.571010 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.580773 (+ 9763us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.588054 (+ 7281us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.590526 (+ 2472us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2947433}
I20260618 13:59:37.593190 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet e025b64380314546af8a1ab90c817825 (table default.Table_f7e1e81727cb4aa8a4e65ddc115b49c0 [id=803d5220ce1a4dcfaf27d93c574be2ac]) successfully deleted
I20260618 13:59:37.593660 29746 tablet_replica.cc:333] T d7a97176f72349cda221baaea61d18ad P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:37.593999 29746 raft_consensus.cc:2243] T d7a97176f72349cda221baaea61d18ad P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.594249 29746 raft_consensus.cc:2272] T d7a97176f72349cda221baaea61d18ad P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.594997 29746 ts_tablet_manager.cc:1916] T d7a97176f72349cda221baaea61d18ad P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.605983 29746 ts_tablet_manager.cc:1929] T d7a97176f72349cda221baaea61d18ad P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.607507 29745 ts_tablet_manager.cc:1929] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.607700 29745 log.cc:1199] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/d667ea5f029f46d89fcb3be03069b97b
I20260618 13:59:37.608150 29745 ts_tablet_manager.cc:1950] T d667ea5f029f46d89fcb3be03069b97b P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:37.609038 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 82) took 4651 ms. Trace:
I20260618 13:59:37.609321 29746 log.cc:1199] T d7a97176f72349cda221baaea61d18ad P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/d7a97176f72349cda221baaea61d18ad
I20260618 13:59:37.609745 29746 ts_tablet_manager.cc:1950] T d7a97176f72349cda221baaea61d18ad P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:37.609851 29745 rpcz_store.cc:276] 0618 13:59:32.957621 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.957802 (+ 181us) service_pool.cc:224] Handling call
0618 13:59:37.588433 (+4630631us) ts_tablet_manager.cc:1151] Deleting tablet d667ea5f029f46d89fcb3be03069b97b
0618 13:59:37.588441 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.597342 (+ 8901us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.607452 (+ 10110us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.609017 (+ 1565us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4630507}
W20260618 13:59:37.611208 28483 consensus_peers.cc:597] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8fd9fa9675234db2abff694a517c0ea8. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:37.611604 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet d667ea5f029f46d89fcb3be03069b97b (table default.Table_f4a03a199e51400ca4c8a5d868ec2025 [id=0ce6f37b68d8435f8f860ba43647bc3b]) successfully deleted
I20260618 13:59:37.611675 29745 tablet_replica.cc:333] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:37.612480 29745 raft_consensus.cc:2243] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.612877 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 97) took 2990 ms. Trace:
I20260618 13:59:37.613148 29745 raft_consensus.cc:2272] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.613878 29745 ts_tablet_manager.cc:1916] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.614020 29746 rpcz_store.cc:276] 0618 13:59:34.622886 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.623602 (+ 716us) service_pool.cc:224] Handling call
0618 13:59:37.593597 (+2969995us) ts_tablet_manager.cc:1151] Deleting tablet d7a97176f72349cda221baaea61d18ad
0618 13:59:37.593612 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.601677 (+ 8065us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.605936 (+ 4259us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.612852 (+ 6916us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2969849}
I20260618 13:59:37.614717 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet d7a97176f72349cda221baaea61d18ad (table default.Table_b0e3e2341f764392af42b949d03dd1f6 [id=27b6a19a28f24f2b8a758bbff06312d0]) successfully deleted
I20260618 13:59:37.615249 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 101) took 2989 ms. Trace:
I20260618 13:59:37.615392 29746 rpcz_store.cc:276] 0618 13:59:34.625430 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.625479 (+ 49us) service_pool.cc:224] Handling call
0618 13:59:37.615135 (+2989656us) ts_tablet_manager.cc:1151] Deleting tablet 897bf6d6bbee4ad89c296325d94a3e53
0618 13:59:37.615148 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.615228 (+ 80us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2989473}
I20260618 13:59:37.616910 30262 raft_consensus.cc:493] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.617028 30262 raft_consensus.cc:515] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.617622 30262 leader_election.cc:290] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:37.618337 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8f124ba79b46e098303b7b2705844d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:37.620072 28346 leader_election.cc:343] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:37.620498 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 897bf6d6bbee4ad89c296325d94a3e53 because the tablet was not found. No further retry: Not found: Tablet not found: 897bf6d6bbee4ad89c296325d94a3e53
I20260618 13:59:37.620842 29746 tablet_replica.cc:333] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
W20260618 13:59:37.561671 28618 consensus_peers.cc:597] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:37.625978 28618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 64333.9us
I20260618 13:59:37.626179 29746 raft_consensus.cc:2243] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.569692 28797 tablet_bootstrap.cc:492] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:37.626821 28797 ts_tablet_manager.cc:1403] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.228s user 0.002s sys 0.003s
I20260618 13:59:37.627741 28797 raft_consensus.cc:359] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.627951 28797 raft_consensus.cc:385] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.628011 28797 raft_consensus.cc:740] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.628204 28797 consensus_queue.cc:260] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.618971 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8f124ba79b46e098303b7b2705844d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:37.628796 28797 ts_tablet_manager.cc:1434] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:37.629125 28797 tablet_bootstrap.cc:492] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:37.629211 28346 leader_election.cc:343] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.629345 28346 leader_election.cc:304] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.629837 30262 raft_consensus.cc:2749] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:37.626390 29746 raft_consensus.cc:2272] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.631170 29746 ts_tablet_manager.cc:1916] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.632036 30059 raft_consensus.cc:493] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.632153 30059 raft_consensus.cc:515] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.632710 30059 leader_election.cc:290] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.633296 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4c6537144d425f81fd1fafcd034b77" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:37.633466 28403 raft_consensus.cc:2468] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:37.633860 28617 leader_election.cc:304] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:37.634183 30059 raft_consensus.cc:2804] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:37.634261 30059 raft_consensus.cc:493] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:37.634321 30059 raft_consensus.cc:3060] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.634817 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4c6537144d425f81fd1fafcd034b77" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:37.635299 28618 leader_election.cc:343] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.636582 28797 tablet_bootstrap.cc:654] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.636906 30059 raft_consensus.cc:515] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.637231 29745 ts_tablet_manager.cc:1929] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.637354 29745 log.cc:1199] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/b6dd422c05324c989e29b20ae2afabad
I20260618 13:59:37.637739 29745 ts_tablet_manager.cc:1950] T b6dd422c05324c989e29b20ae2afabad P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:37.637842 30059 leader_election.cc:290] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.638542 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 83) took 4680 ms. Trace:
I20260618 13:59:37.638660 29745 rpcz_store.cc:276] 0618 13:59:32.958037 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.958080 (+ 43us) service_pool.cc:224] Handling call
0618 13:59:37.611611 (+4653531us) ts_tablet_manager.cc:1151] Deleting tablet b6dd422c05324c989e29b20ae2afabad
0618 13:59:37.611624 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.623559 (+ 11935us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.637184 (+ 13625us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.638526 (+ 1342us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4653377}
I20260618 13:59:37.639029 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4c6537144d425f81fd1fafcd034b77" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:37.639209 28403 raft_consensus.cc:3060] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.640535 29746 ts_tablet_manager.cc:1929] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.640650 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 84) took 4680 ms. Trace:
I20260618 13:59:37.640754 29745 rpcz_store.cc:276] 0618 13:59:32.958586 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.958626 (+ 40us) service_pool.cc:224] Handling call
0618 13:59:37.639111 (+4680485us) ts_tablet_manager.cc:1151] Deleting tablet 8b88ed7cf79c4929a8704f66691cbed4
0618 13:59:37.639120 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.639348 (+ 228us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4680341}
I20260618 13:59:37.641181 29745 tablet_replica.cc:333] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:37.641417 29745 raft_consensus.cc:2243] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.641592 29745 raft_consensus.cc:2272] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.641772 28403 raft_consensus.cc:2468] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:37.642190 30262 raft_consensus.cc:493] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.642242 30152 raft_consensus.cc:493] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.642294 30262 raft_consensus.cc:515] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.642350 30152 raft_consensus.cc:515] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.642903 30152 leader_election.cc:290] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:37.642971 30262 leader_election.cc:290] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:37.643178 28617 consensus_peers.cc:597] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aa8cf927bed9437086efd8e0f21ba26a. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:37.643342 28617 leader_election.cc:304] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:37.643573 29714 raft_consensus.cc:2804] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:37.643693 29714 raft_consensus.cc:697] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:37.643956 29714 consensus_queue.cc:237] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.644809 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e25027f423144b088dad955c7282921" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:37.645365 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet b6dd422c05324c989e29b20ae2afabad (table default.Table_9424eec16a284672a0fe7ab3d0852b40 [id=070ea48af78346aa8ff43f2bb369730c]) successfully deleted
I20260618 13:59:37.645404 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15b1b603f024f3295eb07c98991076d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:37.645676 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 8b88ed7cf79c4929a8704f66691cbed4 because the tablet was not found. No further retry: Not found: Tablet not found: 8b88ed7cf79c4929a8704f66691cbed4
W20260618 13:59:37.646173 28346 leader_election.cc:343] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:37.646405 28346 leader_election.cc:343] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.644531 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e25027f423144b088dad955c7282921" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:37.645157 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4c6537144d425f81fd1fafcd034b77" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:37.647706 28346 leader_election.cc:343] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.647845 28346 leader_election.cc:304] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.646734 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15b1b603f024f3295eb07c98991076d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:37.649032 30262 raft_consensus.cc:2749] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:37.649549 28346 leader_election.cc:343] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.649667 28346 leader_election.cc:304] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.650115 30262 raft_consensus.cc:2749] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:37.650374 28618 leader_election.cc:343] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.650702 29746 log.cc:1199] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/316f470ea82f42fca48e4e96d49c9574
I20260618 13:59:37.650895 29745 ts_tablet_manager.cc:1916] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.651351 29746 ts_tablet_manager.cc:1950] T 316f470ea82f42fca48e4e96d49c9574 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:37.655897 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 102) took 3030 ms. Trace:
I20260618 13:59:37.656082 29746 rpcz_store.cc:276] 0618 13:59:34.625773 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.625826 (+ 53us) service_pool.cc:224] Handling call
0618 13:59:37.620800 (+2994974us) ts_tablet_manager.cc:1151] Deleting tablet 316f470ea82f42fca48e4e96d49c9574
0618 13:59:37.620809 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.635118 (+ 14309us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.640492 (+ 5374us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.655870 (+ 15378us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2994815}
I20260618 13:59:37.656898 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 316f470ea82f42fca48e4e96d49c9574 (table default.Table_28c746533f45405791fc12912e8fbcd4 [id=5415980263ed4fbea10b969e3b8facb4]) successfully deleted
I20260618 13:59:37.657387 29746 tablet_replica.cc:333] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:37.657558 29303 raft_consensus.cc:493] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.657696 29746 raft_consensus.cc:2243] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.657697 29303 raft_consensus.cc:515] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.657872 29746 raft_consensus.cc:2272] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.658288 29303 leader_election.cc:290] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.659214 29746 ts_tablet_manager.cc:1916] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.659426 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "957670718c8c4b8cb87115cc4a2fa861" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:37.660141 28618 leader_election.cc:343] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.660881 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "957670718c8c4b8cb87115cc4a2fa861" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:37.661108 28403 raft_consensus.cc:2468] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:37.661597 28617 leader_election.cc:304] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.662345 29303 raft_consensus.cc:2804] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:37.662441 29303 raft_consensus.cc:493] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:37.662498 29303 raft_consensus.cc:3060] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.664579 29746 ts_tablet_manager.cc:1929] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:37.664660 29746 log.cc:1199] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/d1baf86497d442dbbbddbc3498bebbee
I20260618 13:59:37.664922 29746 ts_tablet_manager.cc:1950] T d1baf86497d442dbbbddbc3498bebbee P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:37.665156 29303 raft_consensus.cc:515] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.665563 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 103) took 3039 ms. Trace:
I20260618 13:59:37.665633 29746 rpcz_store.cc:276] 0618 13:59:34.626113 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.626155 (+ 42us) service_pool.cc:224] Handling call
0618 13:59:37.657327 (+3031172us) ts_tablet_manager.cc:1151] Deleting tablet d1baf86497d442dbbbddbc3498bebbee
0618 13:59:37.657342 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.661992 (+ 4650us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.664538 (+ 2546us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.665544 (+ 1006us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3031031}
I20260618 13:59:37.666065 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet d1baf86497d442dbbbddbc3498bebbee (table default.Table_dac9c04f74664cce9c065edc3e491bf6 [id=5fe80671a240424da5ef1d7958caba2a]) successfully deleted
I20260618 13:59:37.666251 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "957670718c8c4b8cb87115cc4a2fa861" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:37.666791 28618 leader_election.cc:343] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.667093 29303 leader_election.cc:290] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.667753 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "957670718c8c4b8cb87115cc4a2fa861" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:37.668002 28403 raft_consensus.cc:3060] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.674292 28403 raft_consensus.cc:2468] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:37.674459 29745 ts_tablet_manager.cc:1929] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.674611 29745 log.cc:1199] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/b16f7d3c7837498e996bba1818110642
I20260618 13:59:37.674836 28617 leader_election.cc:304] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.675045 29745 ts_tablet_manager.cc:1950] T b16f7d3c7837498e996bba1818110642 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:37.675246 29303 raft_consensus.cc:2804] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:37.675510 29303 raft_consensus.cc:697] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:37.675812 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 85) took 4716 ms. Trace:
I20260618 13:59:37.675827 29303 consensus_queue.cc:237] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.676002 29745 rpcz_store.cc:276] 0618 13:59:32.958966 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.958999 (+ 33us) service_pool.cc:224] Handling call
0618 13:59:37.641141 (+4682142us) ts_tablet_manager.cc:1151] Deleting tablet b16f7d3c7837498e996bba1818110642
0618 13:59:37.641150 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.654229 (+ 13079us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.674375 (+ 20146us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.675776 (+ 1401us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4682006}
I20260618 13:59:37.676661 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet b16f7d3c7837498e996bba1818110642 (table default.Table_dac9c04f74664cce9c065edc3e491bf6 [id=5fe80671a240424da5ef1d7958caba2a]) successfully deleted
I20260618 13:59:37.676733 29745 tablet_replica.cc:333] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:37.676934 29745 raft_consensus.cc:2243] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.677114 29745 raft_consensus.cc:2272] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.677631 29745 ts_tablet_manager.cc:1916] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.678469 28797 tablet_bootstrap.cc:492] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:37.678622 28797 ts_tablet_manager.cc:1403] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.050s user 0.004s sys 0.000s
I20260618 13:59:37.678838 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 104) took 3052 ms. Trace:
I20260618 13:59:37.678967 29746 rpcz_store.cc:276] 0618 13:59:34.626410 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.626474 (+ 64us) service_pool.cc:224] Handling call
0618 13:59:37.678656 (+3052182us) ts_tablet_manager.cc:1151] Deleting tablet 18d8b609b96a4e81b57f69bff522eeea
0618 13:59:37.678667 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.678816 (+ 149us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3052034}
I20260618 13:59:37.679531 28797 raft_consensus.cc:359] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:37.679595 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 18d8b609b96a4e81b57f69bff522eeea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 18d8b609b96a4e81b57f69bff522eeea already in progress: creating tablet
I20260618 13:59:37.679715 28797 raft_consensus.cc:385] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.679771 28797 raft_consensus.cc:740] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.680035 28797 consensus_queue.cc:260] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.680152 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 105) took 3053 ms. Trace:
I20260618 13:59:37.682071 29746 rpcz_store.cc:276] 0618 13:59:34.626742 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.626795 (+ 53us) service_pool.cc:224] Handling call
0618 13:59:37.680072 (+3053277us) ts_tablet_manager.cc:1151] Deleting tablet 69d5e75b4a8247b29d69d07fad5ca5db
0618 13:59:37.680081 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.680135 (+ 54us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3053116}
W20260618 13:59:37.684254 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 69d5e75b4a8247b29d69d07fad5ca5db because the tablet was not found. No further retry: Not found: Tablet not found: 69d5e75b4a8247b29d69d07fad5ca5db
I20260618 13:59:37.684732 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 106) took 3057 ms. Trace:
I20260618 13:59:37.684890 29746 rpcz_store.cc:276] 0618 13:59:34.627073 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.627115 (+ 42us) service_pool.cc:224] Handling call
0618 13:59:37.684599 (+3057484us) ts_tablet_manager.cc:1151] Deleting tablet 7aa9f65d8a5c46fdb745ee17af946dce
0618 13:59:37.684612 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.684711 (+ 99us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3057320}
W20260618 13:59:37.685460 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 7aa9f65d8a5c46fdb745ee17af946dce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7aa9f65d8a5c46fdb745ee17af946dce already in progress: creating tablet
I20260618 13:59:37.685856 29746 tablet_replica.cc:333] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:37.686564 28596 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:37.688114 29746 raft_consensus.cc:2243] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.688406 29746 raft_consensus.cc:2272] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.689062 29746 ts_tablet_manager.cc:1916] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.694957 28797 ts_tablet_manager.cc:1434] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.016s user 0.002s sys 0.000s
I20260618 13:59:37.695269 28797 tablet_bootstrap.cc:492] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:37.698861 28797 tablet_bootstrap.cc:654] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.699709 29745 ts_tablet_manager.cc:1929] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.699882 29745 log.cc:1199] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/c557afc3abfa4ef188bf248e89f49d87
I20260618 13:59:37.700245 29745 ts_tablet_manager.cc:1950] T c557afc3abfa4ef188bf248e89f49d87 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:37.700945 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 86) took 4741 ms. Trace:
I20260618 13:59:37.701068 29745 rpcz_store.cc:276] 0618 13:59:32.959243 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.959280 (+ 37us) service_pool.cc:224] Handling call
0618 13:59:37.676671 (+4717391us) ts_tablet_manager.cc:1151] Deleting tablet c557afc3abfa4ef188bf248e89f49d87
0618 13:59:37.676684 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.692707 (+ 16023us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.699577 (+ 6870us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.700927 (+ 1350us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4717247}
I20260618 13:59:37.701561 29745 tablet_replica.cc:333] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:37.701551 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet c557afc3abfa4ef188bf248e89f49d87 (table default.Table_fdff0afd87584a53a884c9b76926d6d0 [id=7cd5262564144438b00bde60a6cee1e6]) successfully deleted
I20260618 13:59:37.701936 29745 raft_consensus.cc:2243] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.702108 29745 raft_consensus.cc:2272] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.702517 28798 tablet_bootstrap.cc:492] T c3d523419ff244f085d6eac1a3231973 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:37.702652 28798 ts_tablet_manager.cc:1403] T c3d523419ff244f085d6eac1a3231973 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.316s user 0.010s sys 0.002s
I20260618 13:59:37.702806 29745 ts_tablet_manager.cc:1916] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.702940 29746 ts_tablet_manager.cc:1929] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:37.706868 29746 log.cc:1199] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/0a51e830e3014726b1d1c2608973f036
I20260618 13:59:37.707372 29746 ts_tablet_manager.cc:1950] T 0a51e830e3014726b1d1c2608973f036 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:37.703365 28798 raft_consensus.cc:359] T c3d523419ff244f085d6eac1a3231973 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.709024 28798 raft_consensus.cc:385] T c3d523419ff244f085d6eac1a3231973 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.716873 28458 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:37.719518 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:37.719863 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:37.720623 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:37.721047 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:37.721315 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:37.721524 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:37.721729 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:37.721935 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_c70d6c5b0dc94b33b4c5772edd2b9dca [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 2 got 0
I20260618 13:59:37.722182 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:37.722405 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c881dd3cb66344969310956ad6e83cc0 (table default.Table_0158779f563348c78b3505f98d43b839 [id=b1d85ea3aa7047d285ab7c86da8213fd]). Expected version 1 got 0
I20260618 13:59:37.722658 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:37.722873 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:37.723390 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:37.723729 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:37.723999 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:37.724264 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:37.724587 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:37.725324 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:37.725605 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:37.727423 28798 raft_consensus.cc:740] T c3d523419ff244f085d6eac1a3231973 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.727741 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:37.727285 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 107) took 3099 ms. Trace:
I20260618 13:59:37.728085 29746 rpcz_store.cc:276] 0618 13:59:34.627528 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.627574 (+ 46us) service_pool.cc:224] Handling call
0618 13:59:37.685799 (+3058225us) ts_tablet_manager.cc:1151] Deleting tablet 0a51e830e3014726b1d1c2608973f036
0618 13:59:37.685811 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.696003 (+ 10192us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.702705 (+ 6702us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.727259 (+ 24554us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3058059}
I20260618 13:59:37.728627 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:37.728806 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:37.729097 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 2128dbec33db453ba51070767a15b21e (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]). Expected version 2 got 0
I20260618 13:59:37.729178 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 0a51e830e3014726b1d1c2608973f036 (table default.Table_696ead97d50747228517c265538b25fb [id=716ae493050c4288b9bf041c94523b05]) successfully deleted
I20260618 13:59:37.729629 29746 tablet_replica.cc:333] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:37.729652 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:37.729874 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:37.730050 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:37.730523 29746 raft_consensus.cc:2243] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:37.733206 30027 raft_consensus.cc:493] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.733369 30027 raft_consensus.cc:515] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.733964 29746 raft_consensus.cc:2272] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.734122 30027 leader_election.cc:290] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.734740 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5e04eb43ed4cf493c0f6f08e22baeb" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:37.728570 28798 consensus_queue.cc:260] T c3d523419ff244f085d6eac1a3231973 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
W20260618 13:59:37.735378 28618 leader_election.cc:343] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.735571 28798 ts_tablet_manager.cc:1434] T c3d523419ff244f085d6eac1a3231973 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.033s user 0.002s sys 0.000s
I20260618 13:59:37.736047 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5e04eb43ed4cf493c0f6f08e22baeb" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:37.736259 28403 raft_consensus.cc:2468] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:37.736680 28617 leader_election.cc:304] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.737435 30027 raft_consensus.cc:2804] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:37.737540 30027 raft_consensus.cc:493] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:37.737618 30027 raft_consensus.cc:3060] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.740980 30027 raft_consensus.cc:515] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.742184 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5e04eb43ed4cf493c0f6f08e22baeb" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:37.742872 28618 leader_election.cc:343] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.743402 29303 raft_consensus.cc:493] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.743520 29303 raft_consensus.cc:515] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.743755 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5e04eb43ed4cf493c0f6f08e22baeb" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:37.743918 28404 raft_consensus.cc:3060] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.744062 29303 leader_election.cc:290] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.744742 28798 tablet_bootstrap.cc:492] T 2df9e9e811ec4c2ea68467b1294bb62e P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:37.745872 29746 ts_tablet_manager.cc:1916] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.747347 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2ee746c5024cfc9b31c470f9616e7d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:37.748085 28618 leader_election.cc:343] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.749096 28401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2ee746c5024cfc9b31c470f9616e7d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:37.749238 28401 raft_consensus.cc:2468] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:37.749635 28617 leader_election.cc:304] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.750038 30274 consensus_queue.cc:1048] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:37.750180 29714 raft_consensus.cc:2804] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:37.750257 29714 raft_consensus.cc:493] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:37.750316 29714 raft_consensus.cc:3060] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.751148 30027 leader_election.cc:290] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.751479 28404 raft_consensus.cc:2468] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:37.752192 28617 leader_election.cc:304] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.752768 29714 raft_consensus.cc:515] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.753291 29714 leader_election.cc:290] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.753410 29714 raft_consensus.cc:2804] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:37.753494 29714 raft_consensus.cc:697] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:37.753701 29714 consensus_queue.cc:237] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.755177 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2ee746c5024cfc9b31c470f9616e7d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:37.755903 28618 leader_election.cc:343] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.756740 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2ee746c5024cfc9b31c470f9616e7d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:37.762580 29303 raft_consensus.cc:493] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.762709 29303 raft_consensus.cc:515] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.763339 29303 leader_election.cc:290] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.763437 29303 raft_consensus.cc:493] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.763520 29303 raft_consensus.cc:515] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.763629 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ba9186b3ff47d5a2b158fef619686e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:37.763983 29303 leader_election.cc:290] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.764219 28798 tablet_bootstrap.cc:654] T 2df9e9e811ec4c2ea68467b1294bb62e P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:37.764588 28618 leader_election.cc:343] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.765062 28401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ba9186b3ff47d5a2b158fef619686e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:37.765228 28401 raft_consensus.cc:2468] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:37.764639 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa24afe9db46408cace50f642f879f4b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:37.765851 28617 leader_election.cc:304] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.766109 28401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa24afe9db46408cace50f642f879f4b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:37.766247 28401 raft_consensus.cc:2468] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:37.766815 30274 raft_consensus.cc:2804] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:37.766888 30274 raft_consensus.cc:493] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:37.766947 30274 raft_consensus.cc:3060] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.767473 30262 consensus_queue.cc:1048] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:37.769749 30274 raft_consensus.cc:515] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.771072 28617 leader_election.cc:304] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
W20260618 13:59:37.771855 28618 leader_election.cc:343] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.771894 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ba9186b3ff47d5a2b158fef619686e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:37.772233 30274 leader_election.cc:290] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.772416 30274 raft_consensus.cc:2804] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:37.772490 30274 raft_consensus.cc:493] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:37.772562 30274 raft_consensus.cc:3060] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:37.773658 28618 leader_election.cc:343] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.775777 30274 raft_consensus.cc:515] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.776409 28401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ba9186b3ff47d5a2b158fef619686e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:37.776579 28401 raft_consensus.cc:3060] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.778542 29745 ts_tablet_manager.cc:1929] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:37.778633 29745 log.cc:1199] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/5c75278db2234b11bf95f1cf5cc76edd
I20260618 13:59:37.778941 29745 ts_tablet_manager.cc:1950] T 5c75278db2234b11bf95f1cf5cc76edd P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:37.779618 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 87) took 4820 ms. Trace:
I20260618 13:59:37.779752 28401 raft_consensus.cc:2468] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:37.780563 29759 raft_consensus.cc:493] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.780682 29759 raft_consensus.cc:515] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.781229 29759 leader_election.cc:290] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.781538 28403 raft_consensus.cc:3060] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.782570 28401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d3d363766546c4a97b0763bad11f71" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:37.785183 28798 tablet_bootstrap.cc:492] T 2df9e9e811ec4c2ea68467b1294bb62e P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:37.785553 28798 ts_tablet_manager.cc:1403] T 2df9e9e811ec4c2ea68467b1294bb62e P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.047s user 0.003s sys 0.000s
I20260618 13:59:37.785745 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa24afe9db46408cace50f642f879f4b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:37.786214 28618 leader_election.cc:343] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.781219 28617 leader_election.cc:304] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.782912 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d3d363766546c4a97b0763bad11f71" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:37.786860 29714 raft_consensus.cc:2804] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:37.786895 28484 leader_election.cc:304] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.787150 29714 raft_consensus.cc:697] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:37.787314 29759 raft_consensus.cc:2749] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:37.787365 29714 consensus_queue.cc:237] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.788662 28401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa24afe9db46408cace50f642f879f4b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:37.788863 28401 raft_consensus.cc:3060] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.788774 28798 raft_consensus.cc:359] T 2df9e9e811ec4c2ea68467b1294bb62e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.789374 28798 raft_consensus.cc:385] T 2df9e9e811ec4c2ea68467b1294bb62e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.789542 28798 raft_consensus.cc:740] T 2df9e9e811ec4c2ea68467b1294bb62e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.789875 28798 consensus_queue.cc:260] T 2df9e9e811ec4c2ea68467b1294bb62e P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.791335 28798 ts_tablet_manager.cc:1434] T 2df9e9e811ec4c2ea68467b1294bb62e P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260618 13:59:37.792303 28798 tablet_bootstrap.cc:492] T 015494374645469785f1f978eaf0e9a5 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:37.779690 29745 rpcz_store.cc:276] 0618 13:59:32.959538 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.959580 (+ 42us) service_pool.cc:224] Handling call
0618 13:59:37.701518 (+4741938us) ts_tablet_manager.cc:1151] Deleting tablet 5c75278db2234b11bf95f1cf5cc76edd
0618 13:59:37.701528 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.732044 (+ 30516us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.778491 (+ 46447us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.779599 (+ 1108us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4741807}
I20260618 13:59:37.793399 29745 tablet_replica.cc:333] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:37.794728 29745 raft_consensus.cc:2243] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.795984 29746 ts_tablet_manager.cc:1929] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.797143 30274 leader_election.cc:290] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.798712 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 5c75278db2234b11bf95f1cf5cc76edd (table default.Table_9d7999641cc344f18ccc672c49518192 [id=6ac90f0f8ac84ed5953d37156d9b768b]) successfully deleted
I20260618 13:59:37.799031 29745 raft_consensus.cc:2272] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.799067 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:37.799593 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:37.800141 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 72c04f55890648f0a8184f3783243f7c (table default.Table_773be314d7e741fcb4fb6eb7d954802e [id=0d83fc68255c458587390c58b12e00d7]). Expected version 2 got 0
I20260618 13:59:37.801433 29745 ts_tablet_manager.cc:1916] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.801515 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:37.801991 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a]). Expected version 1 got 0
I20260618 13:59:37.802486 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:37.803575 29026 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:37.797972 28403 raft_consensus.cc:2468] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
W20260618 13:59:37.803474 28346 consensus_peers.cc:597] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:37.806490 29746 log.cc:1199] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/8b59255b14df4158afe37cb8890e7415
I20260618 13:59:37.801162 28798 tablet_bootstrap.cc:654] T 015494374645469785f1f978eaf0e9a5 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.807961 29746 ts_tablet_manager.cc:1950] T 8b59255b14df4158afe37cb8890e7415 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:37.808589 28617 leader_election.cc:304] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.808761 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 100) took 3183 ms. Trace:
I20260618 13:59:37.808889 29746 rpcz_store.cc:276] 0618 13:59:34.625300 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.627767 (+ 2467us) service_pool.cc:224] Handling call
0618 13:59:37.729564 (+3101797us) ts_tablet_manager.cc:1151] Deleting tablet 8b59255b14df4158afe37cb8890e7415
0618 13:59:37.729579 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.776944 (+ 47365us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.795930 (+ 18986us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.808738 (+ 12808us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3101645}
I20260618 13:59:37.809293 30059 raft_consensus.cc:2804] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:37.809449 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 8b59255b14df4158afe37cb8890e7415 (table default.Table_74c0d9b4f3724e278144c5fc30d8c2d7 [id=3bcc877eddec4d00b3dc1760cc635b36]) successfully deleted
I20260618 13:59:37.809712 30059 raft_consensus.cc:697] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:37.810024 30059 consensus_queue.cc:237] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.812554 29759 raft_consensus.cc:493] T b95bdb0e36584d92b9c28de257553827 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.812716 29759 raft_consensus.cc:515] T b95bdb0e36584d92b9c28de257553827 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.813500 29759 leader_election.cc:290] T b95bdb0e36584d92b9c28de257553827 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.815188 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b95bdb0e36584d92b9c28de257553827" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:37.816182 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b95bdb0e36584d92b9c28de257553827" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:37.817270 28483 leader_election.cc:304] T b95bdb0e36584d92b9c28de257553827 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.817976 29759 raft_consensus.cc:2749] T b95bdb0e36584d92b9c28de257553827 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:37.825718 29746 tablet_replica.cc:333] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:37.826117 29746 raft_consensus.cc:2243] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.828475 29746 raft_consensus.cc:2272] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.829259 29746 ts_tablet_manager.cc:1916] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.797750 28401 raft_consensus.cc:2468] T fa24afe9db46408cace50f642f879f4b P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:37.833065 28617 leader_election.cc:304] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.833563 30274 raft_consensus.cc:2804] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:37.833760 30274 raft_consensus.cc:697] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:37.834050 30274 consensus_queue.cc:237] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.837397 28798 tablet_bootstrap.cc:492] T 015494374645469785f1f978eaf0e9a5 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:37.837569 28798 ts_tablet_manager.cc:1403] T 015494374645469785f1f978eaf0e9a5 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.045s user 0.003s sys 0.000s
I20260618 13:59:37.838253 28798 raft_consensus.cc:359] T 015494374645469785f1f978eaf0e9a5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.838490 28798 raft_consensus.cc:385] T 015494374645469785f1f978eaf0e9a5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.838615 28798 raft_consensus.cc:740] T 015494374645469785f1f978eaf0e9a5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.838994 28798 consensus_queue.cc:260] T 015494374645469785f1f978eaf0e9a5 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.842280 28798 ts_tablet_manager.cc:1434] T 015494374645469785f1f978eaf0e9a5 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260618 13:59:37.842748 28798 tablet_bootstrap.cc:492] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:37.845547 28798 tablet_bootstrap.cc:654] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.845777 28548 raft_consensus.cc:3060] T a10e83d26b704201b95e7c0c44a1a4f5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.852757 29024 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 106) took 3462 ms. Trace:
I20260618 13:59:37.852921 29024 rpcz_store.cc:276] 0618 13:59:34.390388 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:37.529579 (+3139191us) service_pool.cc:224] Handling call
0618 13:59:37.530611 (+ 1032us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:37.531069 (+ 458us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:37.852501 (+321432us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:37.852603 (+ 102us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:37.852713 (+ 110us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:37.840941 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:37.841076 (+ 135us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:37.841081 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:37.841083 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:37.841174 (+ 91us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:37.841186 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:37.841190 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:37.841265 (+ 75us) tablet.cc:701] Row locks acquired
0618 13:59:37.841267 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:37.841312 (+ 45us) write_op.cc:270] Start()
0618 13:59:37.841352 (+ 40us) write_op.cc:276] Timestamp: P: 1781791177841275 usec, L: 0
0618 13:59:37.841356 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:37.841525 (+ 169us) log.cc:844] Serialized 433 byte log entry
0618 13:59:37.851751 (+ 10226us) op_driver.cc:464] REPLICATION: finished
0618 13:59:37.851953 (+ 202us) write_op.cc:301] APPLY: starting
0618 13:59:37.851967 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:37.852005 (+ 38us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:37.852030 (+ 25us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:37.852125 (+ 95us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:37.852164 (+ 39us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:37.852176 (+ 12us) write_op.cc:312] APPLY: finished
0618 13:59:37.852250 (+ 74us) log.cc:844] Serialized 29 byte log entry
0618 13:59:37.852310 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:37.852327 (+ 17us) write_op.cc:454] Released schema lock
0618 13:59:37.852333 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:37.852402 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:37.852406 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":150,"apply.run_cpu_time_us":439,"apply.run_wall_time_us":439,"num_ops":2,"prepare.queue_time_us":309393,"prepare.run_cpu_time_us":711,"prepare.run_wall_time_us":11024,"raft.queue_time_us":164,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":53,"replication_time_us":10357}]]}
I20260618 13:59:37.854350 29745 ts_tablet_manager.cc:1929] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.854481 29745 log.cc:1199] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/0fae928833e44d2ba860d6184d4c0142
I20260618 13:59:37.854846 29745 ts_tablet_manager.cc:1950] T 0fae928833e44d2ba860d6184d4c0142 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:37.855583 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 79) took 4898 ms. Trace:
I20260618 13:59:37.855660 29745 rpcz_store.cc:276] 0618 13:59:32.957021 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.957094 (+ 73us) service_pool.cc:224] Handling call
0618 13:59:37.793352 (+4836258us) ts_tablet_manager.cc:1151] Deleting tablet 0fae928833e44d2ba860d6184d4c0142
0618 13:59:37.793362 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.828203 (+ 34841us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.854294 (+ 26091us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.855564 (+ 1270us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4821988}
I20260618 13:59:37.856076 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 0fae928833e44d2ba860d6184d4c0142 (table default.Table_9e1e848ca5c94952ab3928c5feafe2f1 [id=14652783e7f54ebcb5d7ba01d1855118]) successfully deleted
I20260618 13:59:37.856418 29745 tablet_replica.cc:333] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:37.856607 29745 raft_consensus.cc:2243] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.856734 29745 raft_consensus.cc:2272] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.857241 29745 ts_tablet_manager.cc:1916] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.858014 29746 ts_tablet_manager.cc:1929] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.858127 29746 log.cc:1199] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/36b8a46a41e543c58a946b6764618c7d
I20260618 13:59:37.858464 29746 ts_tablet_manager.cc:1950] T 36b8a46a41e543c58a946b6764618c7d P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:37.859107 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 109) took 3217 ms. Trace:
I20260618 13:59:37.860461 29746 rpcz_store.cc:276] 0618 13:59:34.641887 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.641947 (+ 60us) service_pool.cc:224] Handling call
0618 13:59:37.825654 (+3183707us) ts_tablet_manager.cc:1151] Deleting tablet 36b8a46a41e543c58a946b6764618c7d
0618 13:59:37.825668 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.832232 (+ 6564us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.857962 (+ 25730us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.859089 (+ 1127us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3183492}
I20260618 13:59:37.861073 29746 tablet_replica.cc:333] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:37.861210 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 36b8a46a41e543c58a946b6764618c7d (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]) successfully deleted
I20260618 13:59:37.863600 28799 tablet_bootstrap.cc:492] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:37.863770 28799 ts_tablet_manager.cc:1403] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.349s user 0.037s sys 0.100s
I20260618 13:59:37.864564 28799 raft_consensus.cc:359] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.865285 29746 raft_consensus.cc:2243] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.865408 28799 raft_consensus.cc:385] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.865546 28799 raft_consensus.cc:740] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.865640 29746 raft_consensus.cc:2272] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.865836 28799 consensus_queue.cc:260] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.866365 28799 ts_tablet_manager.cc:1434] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:37.866686 28799 tablet_bootstrap.cc:492] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:37.866669 29746 ts_tablet_manager.cc:1916] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.869019 28799 tablet_bootstrap.cc:654] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.874039 28797 tablet_bootstrap.cc:492] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:37.874229 28797 ts_tablet_manager.cc:1403] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.179s user 0.003s sys 0.001s
I20260618 13:59:37.875039 28797 raft_consensus.cc:359] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.875221 28797 raft_consensus.cc:385] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.875319 28797 raft_consensus.cc:740] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.875607 28797 consensus_queue.cc:260] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.876137 28797 ts_tablet_manager.cc:1434] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 13:59:37.882712 29644 raft_consensus.cc:493] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.882890 29644 raft_consensus.cc:515] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.883620 29644 leader_election.cc:290] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.884332 29759 raft_consensus.cc:493] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.884485 29759 raft_consensus.cc:515] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.885066 29759 leader_election.cc:290] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.885694 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d046e0421142cd84c5b12d6a434f53" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:37.886353 28618 leader_election.cc:343] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.889314 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4cfca9e76e641a7bc0b58b9ee240f36" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:37.889510 28680 raft_consensus.cc:2468] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:37.889961 28484 leader_election.cc:304] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:37.890245 29759 raft_consensus.cc:2804] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:37.890391 29759 raft_consensus.cc:493] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:37.890499 29759 raft_consensus.cc:3060] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.891731 28798 tablet_bootstrap.cc:492] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:37.891908 28798 ts_tablet_manager.cc:1403] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.049s user 0.007s sys 0.000s
I20260618 13:59:37.892724 28798 raft_consensus.cc:359] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.892956 28798 raft_consensus.cc:385] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.892992 29759 raft_consensus.cc:515] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.893054 28798 raft_consensus.cc:740] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.893368 28798 consensus_queue.cc:260] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.893464 28799 tablet_bootstrap.cc:492] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:37.893576 28799 ts_tablet_manager.cc:1403] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.014s
I20260618 13:59:37.893989 29759 leader_election.cc:290] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.894599 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4cfca9e76e641a7bc0b58b9ee240f36" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:37.894807 28680 raft_consensus.cc:3060] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.896631 28798 ts_tablet_manager.cc:1434] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260618 13:59:37.896943 28798 tablet_bootstrap.cc:492] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:37.897289 28680 raft_consensus.cc:2468] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:37.897686 28484 leader_election.cc:304] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:37.898010 29759 raft_consensus.cc:2804] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:37.898011 28799 raft_consensus.cc:359] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.898241 29759 raft_consensus.cc:697] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:37.898531 29759 consensus_queue.cc:237] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.899111 28799 raft_consensus.cc:385] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.900651 28799 raft_consensus.cc:740] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.900906 28799 consensus_queue.cc:260] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.903635 30059 consensus_queue.cc:1048] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:37.904582 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d046e0421142cd84c5b12d6a434f53" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:37.904736 28402 raft_consensus.cc:2468] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:37.905529 28799 ts_tablet_manager.cc:1434] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.012s user 0.000s sys 0.001s
I20260618 13:59:37.905895 28799 tablet_bootstrap.cc:492] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:37.908020 28799 tablet_bootstrap.cc:654] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.916594 30151 raft_consensus.cc:493] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.916718 30151 raft_consensus.cc:515] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.917254 30151 leader_election.cc:290] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:37.918339 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c74d3ca66f42ad83ef54d9459b99fd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:37.918920 28346 leader_election.cc:343] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.919215 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c74d3ca66f42ad83ef54d9459b99fd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:37.919709 28346 leader_election.cc:343] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.919816 28346 leader_election.cc:304] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.920199 30151 raft_consensus.cc:2749] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:37.921859 29746 ts_tablet_manager.cc:1929] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:37.922065 29746 log.cc:1199] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/5c75278db2234b11bf95f1cf5cc76edd
I20260618 13:59:37.922406 29746 ts_tablet_manager.cc:1950] T 5c75278db2234b11bf95f1cf5cc76edd P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:37.923293 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 110) took 3280 ms. Trace:
I20260618 13:59:37.923461 29746 rpcz_store.cc:276] 0618 13:59:34.642405 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.642482 (+ 77us) service_pool.cc:224] Handling call
0618 13:59:37.861032 (+3218550us) ts_tablet_manager.cc:1151] Deleting tablet 5c75278db2234b11bf95f1cf5cc76edd
0618 13:59:37.861041 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.891836 (+ 30795us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.921771 (+ 29935us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.923272 (+ 1501us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3218377}
I20260618 13:59:37.924018 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 5c75278db2234b11bf95f1cf5cc76edd (table default.Table_9d7999641cc344f18ccc672c49518192 [id=6ac90f0f8ac84ed5953d37156d9b768b]) successfully deleted
I20260618 13:59:37.924351 29746 tablet_replica.cc:333] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:37.924552 29746 raft_consensus.cc:2243] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.924803 29745 ts_tablet_manager.cc:1929] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:37.924932 29745 log.cc:1199] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/d4667fe48f0f4eff9c6c8baabdab58d4
I20260618 13:59:37.925227 29745 ts_tablet_manager.cc:1950] T d4667fe48f0f4eff9c6c8baabdab58d4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:37.926133 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 77) took 4969 ms. Trace:
I20260618 13:59:37.926265 29745 rpcz_store.cc:276] 0618 13:59:32.956824 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.957059 (+ 235us) service_pool.cc:224] Handling call
0618 13:59:37.856367 (+4899308us) ts_tablet_manager.cc:1151] Deleting tablet d4667fe48f0f4eff9c6c8baabdab58d4
0618 13:59:37.856378 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.871396 (+ 15018us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.924738 (+ 53342us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.926112 (+ 1374us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4884841}
I20260618 13:59:37.927834 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet d4667fe48f0f4eff9c6c8baabdab58d4 (table default.Table_0a85d79af65f48a3803bfd1248593a11 [id=2f4d62be772a421ab777fd23be58b719]) successfully deleted
I20260618 13:59:37.928072 29745 tablet_replica.cc:333] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:37.928110 28617 leader_election.cc:304] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:37.928476 29714 raft_consensus.cc:2804] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:37.928642 29714 raft_consensus.cc:493] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:37.928785 29714 raft_consensus.cc:3060] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.929035 29745 raft_consensus.cc:2243] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.929244 29745 raft_consensus.cc:2272] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.930233 29745 ts_tablet_manager.cc:1916] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.931936 29714 raft_consensus.cc:515] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.933221 29714 leader_election.cc:290] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.926241 30165 raft_consensus.cc:493] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.937086 30165 raft_consensus.cc:515] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.937669 30165 leader_election.cc:290] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.932948 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d046e0421142cd84c5b12d6a434f53" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:37.937109 29746 raft_consensus.cc:2272] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:37.938584 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591465e2b57247a2b224cc8b09225811" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
W20260618 13:59:37.941095 28618 leader_election.cc:343] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.943190 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591465e2b57247a2b224cc8b09225811" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:37.943778 28484 leader_election.cc:304] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.944059 29746 ts_tablet_manager.cc:1916] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.946869 28799 tablet_bootstrap.cc:492] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:37.947063 28799 ts_tablet_manager.cc:1403] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.041s user 0.005s sys 0.005s
I20260618 13:59:37.947784 28799 raft_consensus.cc:359] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.947996 28799 raft_consensus.cc:385] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.948088 28799 raft_consensus.cc:740] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.948401 28799 consensus_queue.cc:260] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.948906 29759 raft_consensus.cc:2749] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:37.949347 28618 consensus_peers.cc:597] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4c2b924a096b479cad31c29bf7c99172. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:37.949805 28799 ts_tablet_manager.cc:1434] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260618 13:59:37.898679 28798 tablet_bootstrap.cc:654] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.950138 28799 tablet_bootstrap.cc:492] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:37.951081 28483 leader_election.cc:343] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: c4cfca9e76e641a7bc0b58b9ee240f36
W20260618 13:59:37.951328 28483 leader_election.cc:343] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: c4cfca9e76e641a7bc0b58b9ee240f36
I20260618 13:59:37.954947 28799 tablet_bootstrap.cc:654] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:37.956331 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d046e0421142cd84c5b12d6a434f53" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:37.956528 28403 raft_consensus.cc:3060] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.965449 29745 ts_tablet_manager.cc:1929] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.965560 29745 log.cc:1199] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/2f3d2f236fd44ab8af0524ea282cfa94
I20260618 13:59:37.965930 29745 ts_tablet_manager.cc:1950] T 2f3d2f236fd44ab8af0524ea282cfa94 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:37.966111 30225 catalog_manager.cc:1294] Loaded cluster ID: 7094aaeda1bc41d1ac58a313dfa3bf98
I20260618 13:59:37.966666 30225 catalog_manager.cc:1587] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: loading cluster ID for follower catalog manager: success
I20260618 13:59:37.968472 30059 raft_consensus.cc:493] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.968588 28548 raft_consensus.cc:3060] T 015494374645469785f1f978eaf0e9a5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:37.968626 30059 raft_consensus.cc:515] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.969276 30059 leader_election.cc:290] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:37.972203 28798 tablet_bootstrap.cc:492] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:37.972893 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30f4bd3f86484caab48c1f0806299460" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:37.974241 28798 ts_tablet_manager.cc:1403] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.077s user 0.003s sys 0.000s
W20260618 13:59:37.974502 28618 leader_election.cc:343] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.983547 28798 raft_consensus.cc:359] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.984352 28798 raft_consensus.cc:385] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:37.985119 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 90) took 5013 ms. Trace:
I20260618 13:59:37.985273 28798 raft_consensus.cc:740] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:37.973731 29746 ts_tablet_manager.cc:1929] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:37.985575 29746 log.cc:1199] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/d6871232860b41d489e34d95eb897241
I20260618 13:59:37.986008 29746 ts_tablet_manager.cc:1950] T d6871232860b41d489e34d95eb897241 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:37.987023 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 111) took 3344 ms. Trace:
I20260618 13:59:37.987174 29745 rpcz_store.cc:276] 0618 13:59:32.972057 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.972104 (+ 47us) service_pool.cc:224] Handling call
0618 13:59:37.927819 (+4955715us) ts_tablet_manager.cc:1151] Deleting tablet 2f3d2f236fd44ab8af0524ea282cfa94
0618 13:59:37.928021 (+ 202us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.960420 (+ 32399us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.965394 (+ 4974us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.985102 (+ 19708us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4955245}
I20260618 13:59:37.987702 29745 tablet_replica.cc:333] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:37.987694 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 2f3d2f236fd44ab8af0524ea282cfa94 (table default.Table_04faa8df2e7c49549ccf29cfca0905fa [id=e8a3305c5ad142e98b34b6bba79c9735]) successfully deleted
I20260618 13:59:37.986183 28798 consensus_queue.cc:260] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:37.988443 28798 ts_tablet_manager.cc:1434] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.014s user 0.001s sys 0.000s
I20260618 13:59:37.988744 28798 tablet_bootstrap.cc:492] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:37.990448 29746 rpcz_store.cc:276] 0618 13:59:34.642916 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.643036 (+ 120us) service_pool.cc:224] Handling call
0618 13:59:37.924310 (+3281274us) ts_tablet_manager.cc:1151] Deleting tablet d6871232860b41d489e34d95eb897241
0618 13:59:37.924319 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.968242 (+ 43923us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.973683 (+ 5441us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.987006 (+ 13323us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3281106}
I20260618 13:59:37.981169 30151 raft_consensus.cc:493] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.990826 30151 raft_consensus.cc:515] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:37.991449 30151 leader_election.cc:290] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:37.992499 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "962bf304a285412fb2a529db5b994e9e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:37.993675 28346 leader_election.cc:343] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.994076 29745 raft_consensus.cc:2243] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:37.993208 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "962bf304a285412fb2a529db5b994e9e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:37.995112 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet d6871232860b41d489e34d95eb897241 (table default.Table_c18d75e5991b440caf136e9248a572da [id=5a3cb5befb614a70b4ecf8ccfedf8ecd]) successfully deleted
I20260618 13:59:37.995689 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 112) took 3352 ms. Trace:
I20260618 13:59:37.974740 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30f4bd3f86484caab48c1f0806299460" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:37.995932 28407 raft_consensus.cc:2468] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:37.990098 30152 raft_consensus.cc:493] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:37.996255 30152 raft_consensus.cc:515] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:37.996886 30152 leader_election.cc:290] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:37.994719 28346 leader_election.cc:343] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.997272 28346 leader_election.cc:304] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:37.998371 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b509f940a02545019c526e45e8372c97" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:37.999106 30152 raft_consensus.cc:2749] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:37.999122 29303 consensus_queue.cc:1048] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:37.999406 28346 leader_election.cc:343] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:37.999840 29745 raft_consensus.cc:2272] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.000496 29745 ts_tablet_manager.cc:1916] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:37.998947 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b509f940a02545019c526e45e8372c97" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:38.001514 29746 rpcz_store.cc:276] 0618 13:59:34.643382 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.643431 (+ 49us) service_pool.cc:224] Handling call
0618 13:59:37.995587 (+3352156us) ts_tablet_manager.cc:1151] Deleting tablet 4a01dfb4d59549399c94640f16045555
0618 13:59:37.995600 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.995671 (+ 71us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3352000}
W20260618 13:59:38.002192 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 4a01dfb4d59549399c94640f16045555 because the tablet was not found. No further retry: Not found: Tablet not found: 4a01dfb4d59549399c94640f16045555
I20260618 13:59:38.002583 29746 tablet_replica.cc:333] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
W20260618 13:59:38.001430 28346 leader_election.cc:343] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.003048 28346 leader_election.cc:304] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.003458 30152 raft_consensus.cc:2749] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.002830 29746 raft_consensus.cc:2243] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.005722 29746 raft_consensus.cc:2272] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.005857 28540 raft_consensus.cc:3060] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.007520 28549 raft_consensus.cc:3060] T c3d523419ff244f085d6eac1a3231973 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.011480 28799 tablet_bootstrap.cc:492] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:38.011713 28799 ts_tablet_manager.cc:1403] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.062s user 0.007s sys 0.000s
I20260618 13:59:38.012758 28799 raft_consensus.cc:359] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.012989 28799 raft_consensus.cc:385] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.013093 28799 raft_consensus.cc:740] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.013259 29746 ts_tablet_manager.cc:1916] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.013391 28799 consensus_queue.cc:260] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.014144 28799 ts_tablet_manager.cc:1434] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260618 13:59:38.015532 28618 consensus_peers.cc:597] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:38.016256 28617 consensus_peers.cc:597] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0fae928833e44d2ba860d6184d4c0142. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.013604 28798 tablet_bootstrap.cc:654] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.019714 30225 catalog_manager.cc:1609] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: acquiring CA information for follower catalog manager: success
I20260618 13:59:38.021328 29746 ts_tablet_manager.cc:1929] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:38.022521 29746 log.cc:1199] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/ba0d05ea44e1473fb4b24f61eab096e4
I20260618 13:59:38.023003 29746 ts_tablet_manager.cc:1950] T ba0d05ea44e1473fb4b24f61eab096e4 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:38.022972 28403 raft_consensus.cc:2468] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:38.024184 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 113) took 3380 ms. Trace:
I20260618 13:59:38.025015 29746 rpcz_store.cc:276] 0618 13:59:34.643746 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.643807 (+ 61us) service_pool.cc:224] Handling call
0618 13:59:38.002537 (+3358730us) ts_tablet_manager.cc:1151] Deleting tablet ba0d05ea44e1473fb4b24f61eab096e4
0618 13:59:38.002546 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.017503 (+ 14957us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.021288 (+ 3785us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.024167 (+ 2879us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3358589}
I20260618 13:59:38.025595 29745 ts_tablet_manager.cc:1929] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:38.025687 29745 log.cc:1199] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/ca2ee746c5024cfc9b31c470f9616e7d
I20260618 13:59:38.025954 29745 ts_tablet_manager.cc:1950] T ca2ee746c5024cfc9b31c470f9616e7d P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.027007 28617 leader_election.cc:304] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:38.027534 28617 leader_election.cc:304] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:38.028282 29303 raft_consensus.cc:2804] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:38.028427 29303 raft_consensus.cc:697] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:38.028653 29303 consensus_queue.cc:237] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.035099 29746 tablet_replica.cc:333] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:38.035244 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet ba0d05ea44e1473fb4b24f61eab096e4 (table default.Table_e9839256914f48e6bb6480c5db7eb379 [id=7d40d815ea4544aea591ca23b96b5dd1]) successfully deleted
I20260618 13:59:38.035291 29746 raft_consensus.cc:2243] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:38.035589 29746 raft_consensus.cc:2272] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.036091 29746 ts_tablet_manager.cc:1916] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.033690 28798 tablet_bootstrap.cc:492] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:38.037716 28798 ts_tablet_manager.cc:1403] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.049s user 0.006s sys 0.000s
I20260618 13:59:38.038455 28798 raft_consensus.cc:359] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:38.038616 28798 raft_consensus.cc:385] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.038676 28798 raft_consensus.cc:740] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.038885 28798 consensus_queue.cc:260] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:38.039374 28798 ts_tablet_manager.cc:1434] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:38.039827 28798 tablet_bootstrap.cc:492] T 8ed88c142d7d40cda6392c06bc0486ca P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:38.041324 28798 tablet_bootstrap.cc:654] T 8ed88c142d7d40cda6392c06bc0486ca P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.042484 30027 raft_consensus.cc:2804] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:38.042591 30027 raft_consensus.cc:493] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:38.042697 30027 raft_consensus.cc:3060] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.045598 30027 raft_consensus.cc:515] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.046566 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30f4bd3f86484caab48c1f0806299460" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:38.047086 28618 leader_election.cc:343] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.049585 30225 catalog_manager.cc:1637] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 13:59:38.049959 29746 ts_tablet_manager.cc:1929] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.050124 29746 log.cc:1199] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/eead369de58f40378254697cbbbc6958
I20260618 13:59:38.050576 29746 ts_tablet_manager.cc:1950] T eead369de58f40378254697cbbbc6958 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:38.050833 28797 tablet_bootstrap.cc:492] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:38.051379 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 114) took 3407 ms. Trace:
I20260618 13:59:38.051508 29746 rpcz_store.cc:276] 0618 13:59:34.644071 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.644181 (+ 110us) service_pool.cc:224] Handling call
0618 13:59:38.035053 (+3390872us) ts_tablet_manager.cc:1151] Deleting tablet eead369de58f40378254697cbbbc6958
0618 13:59:38.035062 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.046598 (+ 11536us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.049915 (+ 3317us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.051363 (+ 1448us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3390723}
I20260618 13:59:38.052073 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet eead369de58f40378254697cbbbc6958 (table default.Table_e8ae99dfd8e84d7bb2723eb5fe129c2f [id=6d8f3597814241f3bc8c920d3a59008e]) successfully deleted
I20260618 13:59:38.052523 28797 tablet_bootstrap.cc:654] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.053741 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 84) took 5952 ms. Trace:
I20260618 13:59:38.054004 29566 rpcz_store.cc:276] 0618 13:59:32.101651 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.101698 (+ 47us) service_pool.cc:224] Handling call
0618 13:59:37.425999 (+5324301us) ts_tablet_manager.cc:1151] Deleting tablet 516421cd843b44769efa2b555697c065
0618 13:59:37.426012 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:37.538839 (+112827us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:37.541807 (+ 2968us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.053690 (+511883us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5324156}
I20260618 13:59:38.054665 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 516421cd843b44769efa2b555697c065 (table default.Table_c5c65f038c8f4f2783a705ffcc320629 [id=b06c685b8107494392219d12540a3a6e]) successfully deleted
I20260618 13:59:38.055225 29566 tablet_replica.cc:333] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:38.055706 29566 raft_consensus.cc:2243] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:38.055737 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 115) took 3410 ms. Trace:
I20260618 13:59:38.056026 29746 rpcz_store.cc:276] 0618 13:59:34.645319 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.645364 (+ 45us) service_pool.cc:224] Handling call
0618 13:59:38.052409 (+3407045us) ts_tablet_manager.cc:1151] Deleting tablet edc72a06ee8e4420b07a17d2feb3fcc2
0618 13:59:38.052418 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.055720 (+ 3302us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3406906}
I20260618 13:59:38.056456 29566 raft_consensus.cc:2272] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:38.056542 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet edc72a06ee8e4420b07a17d2feb3fcc2 because the tablet was not found. No further retry: Not found: Tablet not found: edc72a06ee8e4420b07a17d2feb3fcc2
I20260618 13:59:38.056857 29746 tablet_replica.cc:333] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:38.057075 29746 raft_consensus.cc:2243] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.057245 29746 raft_consensus.cc:2272] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.057576 29566 ts_tablet_manager.cc:1916] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.057734 29746 ts_tablet_manager.cc:1916] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.063670 28798 tablet_bootstrap.cc:492] T 8ed88c142d7d40cda6392c06bc0486ca P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:38.063871 28798 ts_tablet_manager.cc:1403] T 8ed88c142d7d40cda6392c06bc0486ca P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.000s
I20260618 13:59:38.064828 28798 raft_consensus.cc:359] T 8ed88c142d7d40cda6392c06bc0486ca P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:38.065074 28798 raft_consensus.cc:385] T 8ed88c142d7d40cda6392c06bc0486ca P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.065166 28798 raft_consensus.cc:740] T 8ed88c142d7d40cda6392c06bc0486ca P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.065408 28798 consensus_queue.cc:260] T 8ed88c142d7d40cda6392c06bc0486ca P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:38.065825 28798 ts_tablet_manager.cc:1434] T 8ed88c142d7d40cda6392c06bc0486ca P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:38.069938 29746 ts_tablet_manager.cc:1929] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:38.070089 29746 log.cc:1199] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/94365c44b5e24acab5df60c67203794a
I20260618 13:59:38.070465 29746 ts_tablet_manager.cc:1950] T 94365c44b5e24acab5df60c67203794a P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:38.071197 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 116) took 3425 ms. Trace:
I20260618 13:59:38.071329 29746 rpcz_store.cc:276] 0618 13:59:34.645625 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.645665 (+ 40us) service_pool.cc:224] Handling call
0618 13:59:38.056814 (+3411149us) ts_tablet_manager.cc:1151] Deleting tablet 94365c44b5e24acab5df60c67203794a
0618 13:59:38.056823 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.066836 (+ 10013us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.069896 (+ 3060us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.071181 (+ 1285us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3411033}
I20260618 13:59:38.071707 29746 tablet_replica.cc:333] T f89c9954d48e407c96c294c26644e25c P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:38.071931 29746 raft_consensus.cc:2243] T f89c9954d48e407c96c294c26644e25c P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.072142 29746 raft_consensus.cc:2272] T f89c9954d48e407c96c294c26644e25c P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.072598 28798 tablet_bootstrap.cc:492] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:38.073558 29759 raft_consensus.cc:493] T 2df9e9e811ec4c2ea68467b1294bb62e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.073719 29759 raft_consensus.cc:515] T 2df9e9e811ec4c2ea68467b1294bb62e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:38.074261 29759 leader_election.cc:290] T 2df9e9e811ec4c2ea68467b1294bb62e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.074445 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 94365c44b5e24acab5df60c67203794a (table default.Table_e4c2db87fb024027b4a6e457a8524f10 [id=770fade529a64ec29e70968dd6d6bb93]) successfully deleted
I20260618 13:59:38.074860 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2df9e9e811ec4c2ea68467b1294bb62e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:38.072649 29746 ts_tablet_manager.cc:1916] T f89c9954d48e407c96c294c26644e25c P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.077565 28798 tablet_bootstrap.cc:654] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.077921 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2df9e9e811ec4c2ea68467b1294bb62e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:38.078462 28484 leader_election.cc:304] T 2df9e9e811ec4c2ea68467b1294bb62e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.078709 29759 raft_consensus.cc:2749] T 2df9e9e811ec4c2ea68467b1294bb62e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.079727 28797 tablet_bootstrap.cc:492] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:38.079955 28797 ts_tablet_manager.cc:1403] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.006s
I20260618 13:59:38.080711 28797 raft_consensus.cc:359] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.080971 28797 raft_consensus.cc:385] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.081090 28797 raft_consensus.cc:740] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.081499 30152 raft_consensus.cc:493] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.081624 30152 raft_consensus.cc:515] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:38.082252 30152 leader_election.cc:290] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:38.082746 28799 tablet_bootstrap.cc:492] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:38.084427 28799 tablet_bootstrap.cc:654] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.085732 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 91) took 5113 ms. Trace:
I20260618 13:59:38.085810 29745 rpcz_store.cc:276] 0618 13:59:32.972153 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.972634 (+ 481us) service_pool.cc:224] Handling call
0618 13:59:37.987658 (+5015024us) ts_tablet_manager.cc:1151] Deleting tablet ca2ee746c5024cfc9b31c470f9616e7d
0618 13:59:37.987668 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.022413 (+ 34745us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.025543 (+ 3130us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.085714 (+ 60171us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5014821}
I20260618 13:59:38.086102 29745 tablet_replica.cc:333] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.086273 29745 raft_consensus.cc:2243] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.086403 29745 raft_consensus.cc:2272] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.086874 29745 ts_tablet_manager.cc:1916] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.089146 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f695a7e6a925470787a708c365e6b86b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:38.090112 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet ca2ee746c5024cfc9b31c470f9616e7d (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]) successfully deleted
W20260618 13:59:38.090277 28346 leader_election.cc:343] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.094174 29745 ts_tablet_manager.cc:1929] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:38.094282 29745 log.cc:1199] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/5e25027f423144b088dad955c7282921
I20260618 13:59:38.081385 28797 consensus_queue.cc:260] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.096524 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f695a7e6a925470787a708c365e6b86b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:38.097079 29745 ts_tablet_manager.cc:1950] T 5e25027f423144b088dad955c7282921 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.097675 28799 tablet_bootstrap.cc:492] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:38.097818 28799 ts_tablet_manager.cc:1403] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.002s
I20260618 13:59:38.097865 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 89) took 5125 ms. Trace:
I20260618 13:59:38.098135 29745 rpcz_store.cc:276] 0618 13:59:32.971917 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.971961 (+ 44us) service_pool.cc:224] Handling call
0618 13:59:38.086064 (+5114103us) ts_tablet_manager.cc:1151] Deleting tablet 5e25027f423144b088dad955c7282921
0618 13:59:38.086072 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.089777 (+ 3705us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.094129 (+ 4352us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.097845 (+ 3716us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5109342}
I20260618 13:59:38.098699 29745 tablet_replica.cc:333] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.098664 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 5e25027f423144b088dad955c7282921 (table default.Table_a7ff69763fdb4ff3b26b180bf05907d9 [id=e39e2c5718794a748361d7b453c65404]) successfully deleted
I20260618 13:59:38.098654 28799 raft_consensus.cc:359] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.098917 28799 raft_consensus.cc:385] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.099009 28799 raft_consensus.cc:740] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.099283 28799 consensus_queue.cc:260] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.099835 28799 ts_tablet_manager.cc:1434] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:38.100056 29745 raft_consensus.cc:2243] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.100221 28799 tablet_bootstrap.cc:492] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:38.101946 28799 tablet_bootstrap.cc:654] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.100221 29745 raft_consensus.cc:2272] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.108258 29745 ts_tablet_manager.cc:1916] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.109212 28797 ts_tablet_manager.cc:1434] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.029s user 0.001s sys 0.001s
I20260618 13:59:38.109566 28797 tablet_bootstrap.cc:492] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:38.110833 28346 leader_election.cc:343] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.110949 28346 leader_election.cc:304] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.111296 30152 raft_consensus.cc:2749] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.111840 28798 tablet_bootstrap.cc:492] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:38.112012 28798 ts_tablet_manager.cc:1403] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.003s
I20260618 13:59:38.112717 28798 raft_consensus.cc:359] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:38.115446 28798 raft_consensus.cc:385] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.115556 28798 raft_consensus.cc:740] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.115797 28798 consensus_queue.cc:260] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:38.124130 28798 ts_tablet_manager.cc:1434] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.012s user 0.000s sys 0.001s
I20260618 13:59:38.126467 30152 raft_consensus.cc:493] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.126588 30152 raft_consensus.cc:515] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.127269 30152 leader_election.cc:290] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:38.129781 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9559c5f2c9e94e858093dd2f0865c865" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:38.130352 28346 leader_election.cc:343] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:38.130651 28618 consensus_peers.cc:597] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.131203 30027 leader_election.cc:290] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.131412 30152 raft_consensus.cc:493] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.131541 30152 raft_consensus.cc:515] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.132105 30152 leader_election.cc:290] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:38.132150 28797 tablet_bootstrap.cc:654] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.132936 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30f4bd3f86484caab48c1f0806299460" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:38.133095 28407 raft_consensus.cc:3060] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.133073 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9559c5f2c9e94e858093dd2f0865c865" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:38.133605 28346 leader_election.cc:343] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.133695 28346 leader_election.cc:304] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.134027 30152 raft_consensus.cc:2749] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.134240 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4539f767c054e9e8ebc1f3f56108e7e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:38.134735 28346 leader_election.cc:343] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.134899 29566 ts_tablet_manager.cc:1929] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.135037 29566 log.cc:1199] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/2f5a334d18ba4c06bb358fa86d8af80d
I20260618 13:59:38.135501 29566 ts_tablet_manager.cc:1950] T 2f5a334d18ba4c06bb358fa86d8af80d P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:38.135811 28407 raft_consensus.cc:2468] T 30f4bd3f86484caab48c1f0806299460 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:38.136207 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4539f767c054e9e8ebc1f3f56108e7e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:38.136364 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 85) took 6033 ms. Trace:
I20260618 13:59:38.136519 29566 rpcz_store.cc:276] 0618 13:59:32.102504 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.102973 (+ 469us) service_pool.cc:224] Handling call
0618 13:59:38.055147 (+5952174us) ts_tablet_manager.cc:1151] Deleting tablet 2f5a334d18ba4c06bb358fa86d8af80d
0618 13:59:38.055157 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.063294 (+ 8137us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.126717 (+ 63423us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.136331 (+ 9614us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5951969}
W20260618 13:59:38.136698 28346 leader_election.cc:343] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.136790 28346 leader_election.cc:304] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.137087 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 2f5a334d18ba4c06bb358fa86d8af80d (table default.Table_2cb0539405824600bb10d2712fb17d31 [id=f90028f13e704830a408df1907fdae49]) successfully deleted
I20260618 13:59:38.137161 30152 raft_consensus.cc:2749] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.137527 29566 tablet_replica.cc:333] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:38.138008 29566 raft_consensus.cc:2243] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:38.138259 28798 tablet_bootstrap.cc:492] T 1d8b3cfac0f648cd8b1c51102132c676 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:38.139153 28799 tablet_bootstrap.cc:492] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:38.139346 28799 ts_tablet_manager.cc:1403] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.004s
I20260618 13:59:38.139822 29566 raft_consensus.cc:2272] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.139940 28798 tablet_bootstrap.cc:654] T 1d8b3cfac0f648cd8b1c51102132c676 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.140272 28799 raft_consensus.cc:359] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.140486 28799 raft_consensus.cc:385] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.140587 28799 raft_consensus.cc:740] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.140831 28799 consensus_queue.cc:260] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.141474 28799 ts_tablet_manager.cc:1434] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:38.141837 29566 ts_tablet_manager.cc:1916] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.141559 28540 raft_consensus.cc:3060] T 8ed88c142d7d40cda6392c06bc0486ca P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.144225 28799 tablet_bootstrap.cc:492] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:38.144882 29745 ts_tablet_manager.cc:1929] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:38.144980 29745 log.cc:1199] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/3a0039a463ec4a9c82a21a3a69b565fc
I20260618 13:59:38.145748 29745 ts_tablet_manager.cc:1950] T 3a0039a463ec4a9c82a21a3a69b565fc P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.146548 28799 tablet_bootstrap.cc:654] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.146559 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 78) took 5189 ms. Trace:
I20260618 13:59:38.147665 29745 rpcz_store.cc:276] 0618 13:59:32.956933 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.957221 (+ 288us) service_pool.cc:224] Handling call
0618 13:59:38.098655 (+5141434us) ts_tablet_manager.cc:1151] Deleting tablet 3a0039a463ec4a9c82a21a3a69b565fc
0618 13:59:38.098665 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.114703 (+ 16038us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.144831 (+ 30128us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.146541 (+ 1710us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5121880}
I20260618 13:59:38.148262 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 3a0039a463ec4a9c82a21a3a69b565fc (table default.Table_8c1fdd54393c4ad383e72a07837ead2a [id=6a7e8e3becb64fd5bc583ee08eaa4437]) successfully deleted
I20260618 13:59:38.148700 29745 tablet_replica.cc:333] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.149624 29745 raft_consensus.cc:2243] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.150305 28798 tablet_bootstrap.cc:492] T 1d8b3cfac0f648cd8b1c51102132c676 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:38.150514 28798 ts_tablet_manager.cc:1403] T 1d8b3cfac0f648cd8b1c51102132c676 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.001s
I20260618 13:59:38.150640 29745 raft_consensus.cc:2272] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.151296 28798 raft_consensus.cc:359] T 1d8b3cfac0f648cd8b1c51102132c676 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.151520 28798 raft_consensus.cc:385] T 1d8b3cfac0f648cd8b1c51102132c676 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.151577 28797 tablet_bootstrap.cc:492] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:38.151660 28798 raft_consensus.cc:740] T 1d8b3cfac0f648cd8b1c51102132c676 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.151719 28797 ts_tablet_manager.cc:1403] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.042s user 0.000s sys 0.004s
I20260618 13:59:38.150326 28799 tablet_bootstrap.cc:492] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:38.152289 28799 ts_tablet_manager.cc:1403] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.003s
I20260618 13:59:38.152982 28799 raft_consensus.cc:359] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.153119 28799 raft_consensus.cc:385] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.153169 28799 raft_consensus.cc:740] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.153448 29745 ts_tablet_manager.cc:1916] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.152824 28798 consensus_queue.cc:260] T 1d8b3cfac0f648cd8b1c51102132c676 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.154181 28798 ts_tablet_manager.cc:1434] T 1d8b3cfac0f648cd8b1c51102132c676 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260618 13:59:38.154480 28798 tablet_bootstrap.cc:492] T 4e4c6537144d425f81fd1fafcd034b77 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:38.155396 28799 consensus_queue.cc:260] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.156255 28799 ts_tablet_manager.cc:1434] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260618 13:59:38.156229 29303 raft_consensus.cc:493] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.156379 29303 raft_consensus.cc:515] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.156549 28799 tablet_bootstrap.cc:492] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:38.157161 29303 leader_election.cc:290] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.158171 28799 tablet_bootstrap.cc:654] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.158171 28617 leader_election.cc:304] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:38.158782 30274 raft_consensus.cc:2804] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:38.158932 30274 raft_consensus.cc:697] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:38.159296 30274 consensus_queue.cc:237] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.160902 28798 tablet_bootstrap.cc:654] T 4e4c6537144d425f81fd1fafcd034b77 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.162768 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fa1a27fc9304e18b96cad9f923779f2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:38.164218 28618 leader_election.cc:343] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.167841 28546 raft_consensus.cc:3060] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.173228 28797 raft_consensus.cc:359] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.173571 28797 raft_consensus.cc:385] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.173744 28797 raft_consensus.cc:740] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.174057 28797 consensus_queue.cc:260] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.176062 28797 ts_tablet_manager.cc:1434] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.024s user 0.001s sys 0.001s
I20260618 13:59:38.177775 28797 tablet_bootstrap.cc:492] T d2d54a901e9f4d5c89700979ce0618bf P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:38.181164 30152 consensus_queue.cc:1048] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:38.181660 30152 raft_consensus.cc:493] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.181813 30152 raft_consensus.cc:515] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.182559 30152 leader_election.cc:290] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:38.184706 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c871e7c8f843486da7ba40145c0fed19" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:38.186022 28346 leader_election.cc:343] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.192010 30151 raft_consensus.cc:493] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.192186 30151 raft_consensus.cc:515] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.192924 30151 leader_election.cc:290] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:38.193463 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f5d4296952e48a48756e6f27140da87" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:38.194433 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f5d4296952e48a48756e6f27140da87" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:38.195104 28346 leader_election.cc:343] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:38.195397 28346 leader_election.cc:343] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.195538 28346 leader_election.cc:304] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.195824 30151 raft_consensus.cc:2749] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.196848 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c871e7c8f843486da7ba40145c0fed19" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:38.198004 28346 consensus_peers.cc:597] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:38.198662 28346 leader_election.cc:343] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.198807 28346 leader_election.cc:304] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.199095 30151 raft_consensus.cc:2749] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.107686 29746 ts_tablet_manager.cc:1929] T f89c9954d48e407c96c294c26644e25c P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.199438 29746 log.cc:1199] T f89c9954d48e407c96c294c26644e25c P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/f89c9954d48e407c96c294c26644e25c
I20260618 13:59:38.199956 29746 ts_tablet_manager.cc:1950] T f89c9954d48e407c96c294c26644e25c P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:38.200927 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 117) took 3555 ms. Trace:
I20260618 13:59:38.201243 29746 rpcz_store.cc:276] 0618 13:59:34.645895 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.645933 (+ 38us) service_pool.cc:224] Handling call
0618 13:59:38.071667 (+3425734us) ts_tablet_manager.cc:1151] Deleting tablet f89c9954d48e407c96c294c26644e25c
0618 13:59:38.071676 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.104818 (+ 33142us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.107634 (+ 2816us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.200888 (+ 93254us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3425612}
I20260618 13:59:38.201917 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet f89c9954d48e407c96c294c26644e25c (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]) successfully deleted
I20260618 13:59:38.141733 28548 raft_consensus.cc:3060] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.202473 28797 tablet_bootstrap.cc:654] T d2d54a901e9f4d5c89700979ce0618bf P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.202948 29759 raft_consensus.cc:493] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.203099 29759 raft_consensus.cc:515] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:38.203800 29759 leader_election.cc:290] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.205042 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572d17724d249739be1a1cb9468f909" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:38.205077 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572d17724d249739be1a1cb9468f909" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:38.205936 28484 leader_election.cc:304] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.206354 29759 raft_consensus.cc:2749] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.219038 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fa1a27fc9304e18b96cad9f923779f2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:38.219385 28407 raft_consensus.cc:2468] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:38.224067 30027 raft_consensus.cc:493] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.224263 30027 raft_consensus.cc:515] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.224308 28617 leader_election.cc:304] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:38.224740 29303 raft_consensus.cc:2804] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:38.224861 29303 raft_consensus.cc:493] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:38.224910 30027 leader_election.cc:290] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.224960 29303 raft_consensus.cc:3060] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.228134 29303 raft_consensus.cc:515] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.229395 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fa1a27fc9304e18b96cad9f923779f2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:38.230036 28618 leader_election.cc:343] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.230490 29303 leader_election.cc:290] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.230858 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c92a594cd64982a8972e4a019d1cd8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:38.231545 28618 leader_election.cc:343] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.232721 28540 raft_consensus.cc:3060] T b95bdb0e36584d92b9c28de257553827 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.234050 28797 tablet_bootstrap.cc:492] T d2d54a901e9f4d5c89700979ce0618bf P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:38.234213 28797 ts_tablet_manager.cc:1403] T d2d54a901e9f4d5c89700979ce0618bf P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.057s user 0.001s sys 0.003s
I20260618 13:59:38.234961 28797 raft_consensus.cc:359] T d2d54a901e9f4d5c89700979ce0618bf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.235141 28797 raft_consensus.cc:385] T d2d54a901e9f4d5c89700979ce0618bf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.235215 28797 raft_consensus.cc:740] T d2d54a901e9f4d5c89700979ce0618bf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.235478 28797 consensus_queue.cc:260] T d2d54a901e9f4d5c89700979ce0618bf P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.242372 29746 tablet_replica.cc:333] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:38.242736 29746 raft_consensus.cc:2243] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:38.243196 29746 raft_consensus.cc:2272] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.244050 30165 raft_consensus.cc:493] T 1d8b3cfac0f648cd8b1c51102132c676 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.244201 30165 raft_consensus.cc:515] T 1d8b3cfac0f648cd8b1c51102132c676 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.244930 30165 leader_election.cc:290] T 1d8b3cfac0f648cd8b1c51102132c676 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.245832 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d8b3cfac0f648cd8b1c51102132c676" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:38.245682 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d8b3cfac0f648cd8b1c51102132c676" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:38.246603 28484 leader_election.cc:304] T 1d8b3cfac0f648cd8b1c51102132c676 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.246994 30165 raft_consensus.cc:2749] T 1d8b3cfac0f648cd8b1c51102132c676 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.247783 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c92a594cd64982a8972e4a019d1cd8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:38.248021 28407 raft_consensus.cc:2468] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:38.250304 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fa1a27fc9304e18b96cad9f923779f2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:38.250520 28407 raft_consensus.cc:3060] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.251618 30151 raft_consensus.cc:493] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.251798 30151 raft_consensus.cc:515] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.252511 30151 leader_election.cc:290] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:38.253464 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7766014ee9814f01ae7ac3e849ca3d50" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:38.254043 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7766014ee9814f01ae7ac3e849ca3d50" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:38.254781 28346 leader_election.cc:343] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:38.255002 28346 leader_election.cc:343] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.255136 28346 leader_election.cc:304] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.255579 30262 raft_consensus.cc:2749] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.266652 28797 ts_tablet_manager.cc:1434] T d2d54a901e9f4d5c89700979ce0618bf P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.032s user 0.000s sys 0.002s
I20260618 13:59:38.268146 28797 tablet_bootstrap.cc:492] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:38.269913 28407 raft_consensus.cc:2468] T 9fa1a27fc9304e18b96cad9f923779f2 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:38.271013 29745 ts_tablet_manager.cc:1929] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:38.272130 28797 tablet_bootstrap.cc:654] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.272305 29745 log.cc:1199] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/18d8b609b96a4e81b57f69bff522eeea
I20260618 13:59:38.272697 29745 ts_tablet_manager.cc:1950] T 18d8b609b96a4e81b57f69bff522eeea P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.273485 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 88) took 5301 ms. Trace:
I20260618 13:59:38.274322 29745 rpcz_store.cc:276] 0618 13:59:32.971747 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.971795 (+ 48us) service_pool.cc:224] Handling call
0618 13:59:38.148322 (+5176527us) ts_tablet_manager.cc:1151] Deleting tablet 18d8b609b96a4e81b57f69bff522eeea
0618 13:59:38.148335 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.217510 (+ 69175us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.270922 (+ 53412us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.273452 (+ 2530us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5171386}
I20260618 13:59:38.274914 29745 tablet_replica.cc:333] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.274976 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 18d8b609b96a4e81b57f69bff522eeea (table default.Table_1b32156053d845a5b192d2ccf583cac6 [id=60305da883dd4da2ab74ac31d5462bf0]) successfully deleted
I20260618 13:59:38.275199 29745 raft_consensus.cc:2243] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.275354 29745 raft_consensus.cc:2272] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.277209 29745 ts_tablet_manager.cc:1916] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.277900 30151 raft_consensus.cc:493] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.278066 30151 raft_consensus.cc:515] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.278882 30151 leader_election.cc:290] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:38.279474 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bbb71449d434d54b007d6f0da0796d7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:38.279930 28346 leader_election.cc:343] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.280244 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bbb71449d434d54b007d6f0da0796d7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:38.280704 28617 leader_election.cc:304] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:38.280931 30151 raft_consensus.cc:493] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260618 13:59:38.280974 28346 leader_election.cc:343] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.281117 28346 leader_election.cc:304] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.281121 30151 raft_consensus.cc:515] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.282523 29746 ts_tablet_manager.cc:1916] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.284070 30262 raft_consensus.cc:2749] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.284796 30151 leader_election.cc:290] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:38.285717 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "438663f365e34fab87e34f434920cecf" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:38.286036 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "438663f365e34fab87e34f434920cecf" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:38.286664 28346 leader_election.cc:343] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.287292 28798 tablet_bootstrap.cc:492] T 4e4c6537144d425f81fd1fafcd034b77 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
W20260618 13:59:38.287248 28346 leader_election.cc:343] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.287441 28798 ts_tablet_manager.cc:1403] T 4e4c6537144d425f81fd1fafcd034b77 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.133s user 0.028s sys 0.026s
I20260618 13:59:38.287428 28346 leader_election.cc:304] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.288271 28798 raft_consensus.cc:359] T 4e4c6537144d425f81fd1fafcd034b77 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.288499 28798 raft_consensus.cc:385] T 4e4c6537144d425f81fd1fafcd034b77 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.288602 28798 raft_consensus.cc:740] T 4e4c6537144d425f81fd1fafcd034b77 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.288869 28798 consensus_queue.cc:260] T 4e4c6537144d425f81fd1fafcd034b77 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.289078 28799 tablet_bootstrap.cc:492] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:38.289248 28799 ts_tablet_manager.cc:1403] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.133s user 0.022s sys 0.061s
I20260618 13:59:38.289454 28798 ts_tablet_manager.cc:1434] T 4e4c6537144d425f81fd1fafcd034b77 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:38.289737 28798 tablet_bootstrap.cc:492] T d464dd402d2044469e364ed37fdfb39b P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:38.290177 28799 raft_consensus.cc:359] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.290385 28799 raft_consensus.cc:385] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.290580 28799 raft_consensus.cc:740] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.290854 28799 consensus_queue.cc:260] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.291190 30262 raft_consensus.cc:2749] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.291357 28798 tablet_bootstrap.cc:654] T d464dd402d2044469e364ed37fdfb39b P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.291510 28799 ts_tablet_manager.cc:1434] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:38.291877 28799 tablet_bootstrap.cc:492] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:38.299832 28799 tablet_bootstrap.cc:654] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.300055 30165 raft_consensus.cc:493] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.300199 30165 raft_consensus.cc:515] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.301002 30165 leader_election.cc:290] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:38.301648 28484 leader_election.cc:343] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 125bb6885ab348d885d5a3b4bdd45bc7
I20260618 13:59:38.303009 28798 tablet_bootstrap.cc:492] T d464dd402d2044469e364ed37fdfb39b P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:38.303170 28798 ts_tablet_manager.cc:1403] T d464dd402d2044469e364ed37fdfb39b P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.003s
I20260618 13:59:38.304020 28798 raft_consensus.cc:359] T d464dd402d2044469e364ed37fdfb39b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.304241 28798 raft_consensus.cc:385] T d464dd402d2044469e364ed37fdfb39b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.304330 28798 raft_consensus.cc:740] T d464dd402d2044469e364ed37fdfb39b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.304643 28798 consensus_queue.cc:260] T d464dd402d2044469e364ed37fdfb39b P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.305238 28798 ts_tablet_manager.cc:1434] T d464dd402d2044469e364ed37fdfb39b P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:38.305584 28798 tablet_bootstrap.cc:492] T ab100ebbe61a4321b9e76c730e5d46ed P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:38.307338 28798 tablet_bootstrap.cc:654] T ab100ebbe61a4321b9e76c730e5d46ed P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.307518 30151 raft_consensus.cc:493] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.307726 30151 raft_consensus.cc:515] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:38.308480 30151 leader_election.cc:290] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:38.309228 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98446f40214c4ac984fcfe98310005c5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:38.309445 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98446f40214c4ac984fcfe98310005c5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:38.309873 28346 leader_election.cc:343] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:38.310146 28346 leader_election.cc:343] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.310254 28346 leader_election.cc:304] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.311426 28540 raft_consensus.cc:3060] T 2df9e9e811ec4c2ea68467b1294bb62e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.314576 28798 tablet_bootstrap.cc:492] T ab100ebbe61a4321b9e76c730e5d46ed P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:38.314796 28798 ts_tablet_manager.cc:1403] T ab100ebbe61a4321b9e76c730e5d46ed P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260618 13:59:38.315578 28798 raft_consensus.cc:359] T ab100ebbe61a4321b9e76c730e5d46ed P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.315748 28798 raft_consensus.cc:385] T ab100ebbe61a4321b9e76c730e5d46ed P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.315837 28798 raft_consensus.cc:740] T ab100ebbe61a4321b9e76c730e5d46ed P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.316120 28798 consensus_queue.cc:260] T ab100ebbe61a4321b9e76c730e5d46ed P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.316341 30151 raft_consensus.cc:2749] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.316653 28798 ts_tablet_manager.cc:1434] T ab100ebbe61a4321b9e76c730e5d46ed P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:38.316936 28798 tablet_bootstrap.cc:492] T 9ead386562eb4fd787c7aee776c2194f P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:38.318909 28798 tablet_bootstrap.cc:654] T 9ead386562eb4fd787c7aee776c2194f P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.320055 29303 raft_consensus.cc:2804] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:38.320142 29303 raft_consensus.cc:493] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:38.320199 29303 raft_consensus.cc:3060] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.323007 29303 raft_consensus.cc:515] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.323999 29303 leader_election.cc:290] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.324088 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c92a594cd64982a8972e4a019d1cd8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:38.324661 28618 leader_election.cc:343] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:38.324970 28617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 44323.8us
I20260618 13:59:38.325196 28617 leader_election.cc:304] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:38.325412 29303 raft_consensus.cc:2804] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:38.325631 29303 raft_consensus.cc:697] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:38.325876 29303 consensus_queue.cc:237] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.327327 30165 raft_consensus.cc:493] T ab100ebbe61a4321b9e76c730e5d46ed P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.327509 30165 raft_consensus.cc:515] T ab100ebbe61a4321b9e76c730e5d46ed P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.328122 30165 leader_election.cc:290] T ab100ebbe61a4321b9e76c730e5d46ed P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.328483 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab100ebbe61a4321b9e76c730e5d46ed" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:38.328714 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab100ebbe61a4321b9e76c730e5d46ed" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:38.329291 28483 leader_election.cc:304] T ab100ebbe61a4321b9e76c730e5d46ed P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.329890 29749 raft_consensus.cc:2749] T ab100ebbe61a4321b9e76c730e5d46ed P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.336658 28798 tablet_bootstrap.cc:492] T 9ead386562eb4fd787c7aee776c2194f P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:38.336817 28798 ts_tablet_manager.cc:1403] T 9ead386562eb4fd787c7aee776c2194f P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.000s
I20260618 13:59:38.337585 28798 raft_consensus.cc:359] T 9ead386562eb4fd787c7aee776c2194f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.337767 28798 raft_consensus.cc:385] T 9ead386562eb4fd787c7aee776c2194f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.337837 28798 raft_consensus.cc:740] T 9ead386562eb4fd787c7aee776c2194f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.338063 28798 consensus_queue.cc:260] T 9ead386562eb4fd787c7aee776c2194f P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.338572 28798 ts_tablet_manager.cc:1434] T 9ead386562eb4fd787c7aee776c2194f P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:38.338877 28798 tablet_bootstrap.cc:492] T 4bc87770baa34842a25ed97a19354446 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
W20260618 13:59:38.340066 28618 consensus_peers.cc:597] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.340615 28798 tablet_bootstrap.cc:654] T 4bc87770baa34842a25ed97a19354446 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:38.343204 28483 leader_election.cc:343] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 125bb6885ab348d885d5a3b4bdd45bc7
I20260618 13:59:38.343341 28483 leader_election.cc:304] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.343897 30165 raft_consensus.cc:2749] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:38.354542 28798 tablet_bootstrap.cc:492] T 4bc87770baa34842a25ed97a19354446 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:38.354717 28798 ts_tablet_manager.cc:1403] T 4bc87770baa34842a25ed97a19354446 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.001s
I20260618 13:59:38.355535 28798 raft_consensus.cc:359] T 4bc87770baa34842a25ed97a19354446 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:38.355724 28798 raft_consensus.cc:385] T 4bc87770baa34842a25ed97a19354446 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.355818 28798 raft_consensus.cc:740] T 4bc87770baa34842a25ed97a19354446 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.356101 28798 consensus_queue.cc:260] T 4bc87770baa34842a25ed97a19354446 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:38.356613 28798 ts_tablet_manager.cc:1434] T 4bc87770baa34842a25ed97a19354446 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:38.356905 28798 tablet_bootstrap.cc:492] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:38.357131 30165 raft_consensus.cc:493] T 4e4c6537144d425f81fd1fafcd034b77 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.357342 30165 raft_consensus.cc:515] T 4e4c6537144d425f81fd1fafcd034b77 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.358385 30165 leader_election.cc:290] T 4e4c6537144d425f81fd1fafcd034b77 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.358505 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4c6537144d425f81fd1fafcd034b77" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:38.358814 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4c6537144d425f81fd1fafcd034b77" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:38.359045 28407 raft_consensus.cc:2393] T 4e4c6537144d425f81fd1fafcd034b77 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:38.359465 28483 leader_election.cc:304] T 4e4c6537144d425f81fd1fafcd034b77 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.359742 29749 raft_consensus.cc:3060] T 4e4c6537144d425f81fd1fafcd034b77 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.360782 28798 tablet_bootstrap.cc:654] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.363461 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c92a594cd64982a8972e4a019d1cd8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:38.363639 28403 raft_consensus.cc:3060] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.367412 28403 raft_consensus.cc:2468] T 41c92a594cd64982a8972e4a019d1cd8 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:38.370520 29749 raft_consensus.cc:2749] T 4e4c6537144d425f81fd1fafcd034b77 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.385574 29566 ts_tablet_manager.cc:1929] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.385737 29566 log.cc:1199] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/b16f7d3c7837498e996bba1818110642
I20260618 13:59:38.386142 29566 ts_tablet_manager.cc:1950] T b16f7d3c7837498e996bba1818110642 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:38.389973 28797 tablet_bootstrap.cc:492] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:38.390115 28797 ts_tablet_manager.cc:1403] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.122s user 0.023s sys 0.072s
I20260618 13:59:38.391067 28797 raft_consensus.cc:359] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.391274 28797 raft_consensus.cc:385] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.391337 28797 raft_consensus.cc:740] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.391599 28797 consensus_queue.cc:260] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.397466 29759 raft_consensus.cc:493] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:38.397667 29759 raft_consensus.cc:515] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:38.398515 29759 leader_election.cc:290] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.399055 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09cbe2337dd14da1b8e0629ded7e7ad0" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:38.399742 30151 raft_consensus.cc:493] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.399938 30151 raft_consensus.cc:515] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.400604 30151 leader_election.cc:290] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:38.401036 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3a3395aa90e426eb48a3920a8c45ba8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:38.401347 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09cbe2337dd14da1b8e0629ded7e7ad0" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
W20260618 13:59:38.401571 28346 leader_election.cc:343] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.401921 28483 leader_election.cc:304] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.402555 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3a3395aa90e426eb48a3920a8c45ba8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:38.403045 28346 leader_election.cc:343] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.403136 28346 leader_election.cc:304] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.403510 29749 raft_consensus.cc:2749] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:38.405702 30151 raft_consensus.cc:2749] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.405936 29745 ts_tablet_manager.cc:1929] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:38.406098 29745 log.cc:1199] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/97a9f949999446f09359629fe483bf67
I20260618 13:59:38.406499 29745 ts_tablet_manager.cc:1950] T 97a9f949999446f09359629fe483bf67 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.407325 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 92) took 5421 ms. Trace:
I20260618 13:59:38.407459 29745 rpcz_store.cc:276] 0618 13:59:32.986158 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.986205 (+ 47us) service_pool.cc:224] Handling call
0618 13:59:38.274850 (+5288645us) ts_tablet_manager.cc:1151] Deleting tablet 97a9f949999446f09359629fe483bf67
0618 13:59:38.274860 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.402527 (+127667us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.405885 (+ 3358us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.407298 (+ 1413us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5288482}
I20260618 13:59:38.407910 29745 tablet_replica.cc:333] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.408154 29745 raft_consensus.cc:2243] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.408353 29745 raft_consensus.cc:2272] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.408686 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 97a9f949999446f09359629fe483bf67 (table default.Table_9424eec16a284672a0fe7ab3d0852b40 [id=070ea48af78346aa8ff43f2bb369730c]) successfully deleted
I20260618 13:59:38.408934 29745 ts_tablet_manager.cc:1916] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.415541 29745 ts_tablet_manager.cc:1929] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:38.415743 29745 log.cc:1199] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/e5f7eaa7d218440ca2c35340c5c87814
I20260618 13:59:38.416118 29745 ts_tablet_manager.cc:1950] T e5f7eaa7d218440ca2c35340c5c87814 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.416965 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 93) took 5430 ms. Trace:
I20260618 13:59:38.417119 29745 rpcz_store.cc:276] 0618 13:59:32.986503 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.986545 (+ 42us) service_pool.cc:224] Handling call
0618 13:59:38.407861 (+5421316us) ts_tablet_manager.cc:1151] Deleting tablet e5f7eaa7d218440ca2c35340c5c87814
0618 13:59:38.407871 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.412283 (+ 4412us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.415490 (+ 3207us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.416945 (+ 1455us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5421161}
I20260618 13:59:38.417635 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 94) took 5430 ms. Trace:
I20260618 13:59:38.417770 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet e5f7eaa7d218440ca2c35340c5c87814 (table default.Table_9a78338c3e414b0abea1281082202f57 [id=5697846ccd9c4c659755a0f9b8a66b72]) successfully deleted
I20260618 13:59:38.417775 29745 rpcz_store.cc:276] 0618 13:59:32.986804 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.986843 (+ 39us) service_pool.cc:224] Handling call
0618 13:59:38.417541 (+5430698us) ts_tablet_manager.cc:1151] Deleting tablet 3dd0eaa64271489d92aa380df877f257
0618 13:59:38.417550 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.417618 (+ 68us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5430549}
I20260618 13:59:38.418074 29745 tablet_replica.cc:333] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.418289 29745 raft_consensus.cc:2243] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
W20260618 13:59:38.418295 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 3dd0eaa64271489d92aa380df877f257 because the tablet was not found. No further retry: Not found: Tablet not found: 3dd0eaa64271489d92aa380df877f257
I20260618 13:59:38.418536 29745 raft_consensus.cc:2272] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.419291 29745 ts_tablet_manager.cc:1916] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.419401 28540 raft_consensus.cc:3060] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.423084 28797 ts_tablet_manager.cc:1434] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.033s user 0.003s sys 0.000s
I20260618 13:59:38.423707 30027 raft_consensus.cc:493] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.423834 30027 raft_consensus.cc:515] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.424337 30027 leader_election.cc:290] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.425050 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f83bb8c3c504d5f9affbaa8be3f1508" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:38.425559 28618 leader_election.cc:343] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.426673 29745 ts_tablet_manager.cc:1929] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.426819 29745 log.cc:1199] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/04dac931e62a448d99633878eb70b92f
I20260618 13:59:38.427242 29745 ts_tablet_manager.cc:1950] T 04dac931e62a448d99633878eb70b92f P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.428095 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 95) took 5441 ms. Trace:
I20260618 13:59:38.428225 29745 rpcz_store.cc:276] 0618 13:59:32.987118 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.987159 (+ 41us) service_pool.cc:224] Handling call
0618 13:59:38.418030 (+5430871us) ts_tablet_manager.cc:1151] Deleting tablet 04dac931e62a448d99633878eb70b92f
0618 13:59:38.418041 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.421942 (+ 3901us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.426620 (+ 4678us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.428074 (+ 1454us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5430734}
I20260618 13:59:38.428632 29745 tablet_replica.cc:333] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.428889 29745 raft_consensus.cc:2243] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.428828 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 04dac931e62a448d99633878eb70b92f (table default.Table_f37d21a9a59249d6a0b87c3788ff4873 [id=22eb9037fc66472793e9fd3f03fd7218]) successfully deleted
I20260618 13:59:38.429081 29745 raft_consensus.cc:2272] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.429711 29745 ts_tablet_manager.cc:1916] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:38.431974 28618 consensus_peers.cc:597] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:38.432283 28617 consensus_peers.cc:597] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c557afc3abfa4ef188bf248e89f49d87. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.432457 28617 leader_election.cc:304] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:38.432735 29303 raft_consensus.cc:2804] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:38.432873 29303 raft_consensus.cc:697] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:38.433156 29303 consensus_queue.cc:237] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.436102 29745 ts_tablet_manager.cc:1929] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.436269 29745 log.cc:1199] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/ab521028ac1a4d0da013112d76bb169f
I20260618 13:59:38.436724 29745 ts_tablet_manager.cc:1950] T ab521028ac1a4d0da013112d76bb169f P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.437584 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 96) took 5450 ms. Trace:
I20260618 13:59:38.437738 29745 rpcz_store.cc:276] 0618 13:59:32.987427 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.987466 (+ 39us) service_pool.cc:224] Handling call
0618 13:59:38.428589 (+5441123us) ts_tablet_manager.cc:1151] Deleting tablet ab521028ac1a4d0da013112d76bb169f
0618 13:59:38.428599 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.433014 (+ 4415us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.436052 (+ 3038us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.437563 (+ 1511us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5441000}
I20260618 13:59:38.438169 29745 tablet_replica.cc:333] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.438496 29745 raft_consensus.cc:2243] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.438306 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet ab521028ac1a4d0da013112d76bb169f (table default.Table_3deacb2c9668491a85bbb09ae3f57d66 [id=431a660bfbed46248a603dc7781cf543]) successfully deleted
I20260618 13:59:38.438689 29745 raft_consensus.cc:2272] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.439185 29745 ts_tablet_manager.cc:1916] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.442306 29746 ts_tablet_manager.cc:1929] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.442860 30340 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a950ab7cd66b4cb0b29592aa692c2dd0)
I20260618 13:59:38.443158 30340 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:38.444672 29048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 623 } } ignore_live_leader: false dest_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" is_pre_election: true
I20260618 13:59:38.447422 29745 ts_tablet_manager.cc:1929] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:38.447625 29745 log.cc:1199] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/f3ba9186b3ff47d5a2b158fef619686e
I20260618 13:59:38.448038 29745 ts_tablet_manager.cc:1950] T f3ba9186b3ff47d5a2b158fef619686e P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.448984 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 97) took 5461 ms. Trace:
I20260618 13:59:38.449149 29745 rpcz_store.cc:276] 0618 13:59:32.987693 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.987732 (+ 39us) service_pool.cc:224] Handling call
0618 13:59:38.438126 (+5450394us) ts_tablet_manager.cc:1151] Deleting tablet f3ba9186b3ff47d5a2b158fef619686e
0618 13:59:38.438135 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.442484 (+ 4349us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.447364 (+ 4880us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.448959 (+ 1595us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5450268}
I20260618 13:59:38.449822 29745 tablet_replica.cc:333] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.449828 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet f3ba9186b3ff47d5a2b158fef619686e (table default.Table_3ef2ff183a6044e2a9e069a4eac133b8 [id=60e91b2138bf41a5a8dee65c1874678a]) successfully deleted
I20260618 13:59:38.450136 29745 raft_consensus.cc:2243] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.450388 29745 raft_consensus.cc:2272] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.451109 29745 ts_tablet_manager.cc:1916] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.458657 29745 ts_tablet_manager.cc:1929] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.458854 29745 log.cc:1199] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/c3d523419ff244f085d6eac1a3231973
I20260618 13:59:38.459342 29745 ts_tablet_manager.cc:1950] T c3d523419ff244f085d6eac1a3231973 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.460592 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 98) took 5472 ms. Trace:
I20260618 13:59:38.460753 29745 rpcz_store.cc:276] 0618 13:59:32.988100 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.988304 (+ 204us) service_pool.cc:224] Handling call
0618 13:59:38.449763 (+5461459us) ts_tablet_manager.cc:1151] Deleting tablet c3d523419ff244f085d6eac1a3231973
0618 13:59:38.449776 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.455064 (+ 5288us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.458598 (+ 3534us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.460569 (+ 1971us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5461307}
I20260618 13:59:38.461253 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet c3d523419ff244f085d6eac1a3231973 (table default.Table_176a739c87ee45798e2470f8ec00d632 [id=2534d29e39c14366b10492a7c6a92947]) successfully deleted
I20260618 13:59:38.461341 29745 tablet_replica.cc:333] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.461576 29745 raft_consensus.cc:2243] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.461825 29745 raft_consensus.cc:2272] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.462633 29745 ts_tablet_manager.cc:1916] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.471091 29745 ts_tablet_manager.cc:1929] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.471248 29745 log.cc:1199] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/7572d17724d249739be1a1cb9468f909
I20260618 13:59:38.472009 29745 ts_tablet_manager.cc:1950] T 7572d17724d249739be1a1cb9468f909 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.472913 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 99) took 5484 ms. Trace:
I20260618 13:59:38.473076 29745 rpcz_store.cc:276] 0618 13:59:32.988612 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.988655 (+ 43us) service_pool.cc:224] Handling call
0618 13:59:38.461271 (+5472616us) ts_tablet_manager.cc:1151] Deleting tablet 7572d17724d249739be1a1cb9468f909
0618 13:59:38.461284 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.467622 (+ 6338us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.471052 (+ 3430us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.472892 (+ 1840us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5472469}
I20260618 13:59:38.473732 29745 tablet_replica.cc:333] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.473815 28797 tablet_bootstrap.cc:492] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:38.474073 29745 raft_consensus.cc:2243] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.474323 29745 raft_consensus.cc:2272] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.474996 29745 ts_tablet_manager.cc:1916] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.475502 28797 tablet_bootstrap.cc:654] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.476449 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 7572d17724d249739be1a1cb9468f909 (table default.Table_b583e9f1f2464ebe9df9b4663bb0a4f8 [id=67643f46e18742859d79b0608aeaad23]) successfully deleted
W20260618 13:59:38.476785 28617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 44354us
I20260618 13:59:38.477227 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 86) took 6369 ms. Trace:
I20260618 13:59:38.477370 29566 rpcz_store.cc:276] 0618 13:59:32.107435 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.107484 (+ 49us) service_pool.cc:224] Handling call
0618 13:59:38.137460 (+6029976us) ts_tablet_manager.cc:1151] Deleting tablet b16f7d3c7837498e996bba1818110642
0618 13:59:38.137470 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.198652 (+ 61182us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.385468 (+186816us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.477209 (+ 91741us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6029814}
I20260618 13:59:38.477926 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet b16f7d3c7837498e996bba1818110642 (table default.Table_dac9c04f74664cce9c065edc3e491bf6 [id=5fe80671a240424da5ef1d7958caba2a]) successfully deleted
I20260618 13:59:38.478262 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 87) took 6369 ms. Trace:
I20260618 13:59:38.478379 29566 rpcz_store.cc:276] 0618 13:59:32.108543 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.108589 (+ 46us) service_pool.cc:224] Handling call
0618 13:59:38.478170 (+6369581us) ts_tablet_manager.cc:1151] Deleting tablet c84189a4b1194c87bbdf81f6e0ed56e4
0618 13:59:38.478179 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.478246 (+ 67us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6369441}
W20260618 13:59:38.478869 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet c84189a4b1194c87bbdf81f6e0ed56e4 because the tablet was not found. No further retry: Not found: Tablet not found: c84189a4b1194c87bbdf81f6e0ed56e4
I20260618 13:59:38.479635 28401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f83bb8c3c504d5f9affbaa8be3f1508" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:38.479809 28401 raft_consensus.cc:2468] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:38.480683 29745 ts_tablet_manager.cc:1929] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.480805 29745 log.cc:1199] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/015494374645469785f1f978eaf0e9a5
I20260618 13:59:38.481460 29745 ts_tablet_manager.cc:1950] T 015494374645469785f1f978eaf0e9a5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.482082 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 100) took 5493 ms. Trace:
I20260618 13:59:38.482206 29745 rpcz_store.cc:276] 0618 13:59:32.988968 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.989014 (+ 46us) service_pool.cc:224] Handling call
0618 13:59:38.473676 (+5484662us) ts_tablet_manager.cc:1151] Deleting tablet 015494374645469785f1f978eaf0e9a5
0618 13:59:38.473688 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.478146 (+ 4458us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.480642 (+ 2496us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.482062 (+ 1420us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5484509}
I20260618 13:59:38.483018 29745 tablet_replica.cc:333] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.483156 28799 tablet_bootstrap.cc:492] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:38.483273 29745 raft_consensus.cc:2243] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.483471 29745 raft_consensus.cc:2272] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.483969 29745 ts_tablet_manager.cc:1916] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.484498 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 015494374645469785f1f978eaf0e9a5 (table default.Table_7699960fdee24242bf51bcfcbd6f68c2 [id=3b9ab803536344d3a59ff77b2b58ae31]) successfully deleted
I20260618 13:59:38.483281 28799 ts_tablet_manager.cc:1403] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.192s user 0.023s sys 0.042s
I20260618 13:59:38.487711 28799 raft_consensus.cc:359] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.487895 28799 raft_consensus.cc:385] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.487953 28799 raft_consensus.cc:740] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.488160 28799 consensus_queue.cc:260] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.488684 28799 ts_tablet_manager.cc:1434] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:38.488927 28799 tablet_bootstrap.cc:492] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:38.490474 28799 tablet_bootstrap.cc:654] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.493391 29745 ts_tablet_manager.cc:1929] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.493484 29745 log.cc:1199] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/75824a5d2f7e47838b60e6a2b785c6b2
I20260618 13:59:38.493793 29745 ts_tablet_manager.cc:1950] T 75824a5d2f7e47838b60e6a2b785c6b2 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.494448 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 101) took 5505 ms. Trace:
I20260618 13:59:38.494526 29745 rpcz_store.cc:276] 0618 13:59:32.989486 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.989530 (+ 44us) service_pool.cc:224] Handling call
0618 13:59:38.482980 (+5493450us) ts_tablet_manager.cc:1151] Deleting tablet 75824a5d2f7e47838b60e6a2b785c6b2
0618 13:59:38.482988 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.486665 (+ 3677us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.493347 (+ 6682us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.494413 (+ 1066us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5493308}
I20260618 13:59:38.494794 29745 tablet_replica.cc:333] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.494905 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 75824a5d2f7e47838b60e6a2b785c6b2 (table default.Table_d838306823ff4e22a68efe0a8cb751ff [id=bbfd0a6a47db4b1b8c534405b1068d5e]) successfully deleted
I20260618 13:59:38.494956 29745 raft_consensus.cc:2243] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:38.495187 29745 raft_consensus.cc:2272] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.495625 29745 ts_tablet_manager.cc:1916] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.496243 29566 tablet_replica.cc:333] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:38.496461 29566 raft_consensus.cc:2243] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:38.497152 29566 raft_consensus.cc:2272] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.497289 29303 raft_consensus.cc:493] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.497504 29303 raft_consensus.cc:515] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.499245 29303 leader_election.cc:290] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.500831 29566 ts_tablet_manager.cc:1916] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.503805 29745 ts_tablet_manager.cc:1929] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.504436 28797 tablet_bootstrap.cc:492] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:38.504583 28797 ts_tablet_manager.cc:1403] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.000s
I20260618 13:59:38.505535 28797 raft_consensus.cc:359] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.505738 28797 raft_consensus.cc:385] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.505807 28797 raft_consensus.cc:740] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.505949 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:38.506040 28797 consensus_queue.cc:260] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.506897 28797 ts_tablet_manager.cc:1434] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:38.506639 29025 catalog_manager.cc:5696] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:38.507393 28797 tablet_bootstrap.cc:492] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:38.507349 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:38.508021 29745 log.cc:1199] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/be59f988f0e34b9480f859cf88b9c223
I20260618 13:59:38.508540 29745 ts_tablet_manager.cc:1950] T be59f988f0e34b9480f859cf88b9c223 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.509202 28797 tablet_bootstrap.cc:654] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.509907 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 102) took 5520 ms. Trace:
I20260618 13:59:38.510507 29745 rpcz_store.cc:276] 0618 13:59:32.989840 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.989883 (+ 43us) service_pool.cc:224] Handling call
0618 13:59:38.494755 (+5504872us) ts_tablet_manager.cc:1151] Deleting tablet be59f988f0e34b9480f859cf88b9c223
0618 13:59:38.494765 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.501033 (+ 6268us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.503754 (+ 2721us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.509889 (+ 6135us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5504724}
I20260618 13:59:38.511168 29745 tablet_replica.cc:333] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.511230 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet be59f988f0e34b9480f859cf88b9c223 (table default.Table_b8741a7fd29f4e78a948b1c419595a85 [id=6f4d908593ae40f09e2c94cef218e1fd]) successfully deleted
I20260618 13:59:38.511637 29745 raft_consensus.cc:2243] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:38.512216 29745 raft_consensus.cc:2272] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.513871 29745 ts_tablet_manager.cc:1916] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.516682 29303 consensus_queue.cc:1048] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:38.517333 28617 leader_election.cc:304] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:38.518275 29303 raft_consensus.cc:2804] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:38.518406 29303 raft_consensus.cc:493] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:38.518544 29303 raft_consensus.cc:3060] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.518836 30340 leader_election.cc:290] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a950ab7cd66b4cb0b29592aa692c2dd0 (127.27.100.61:45967), 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083)
I20260618 13:59:38.521202 29303 raft_consensus.cc:515] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.524314 29303 leader_election.cc:290] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.526726 28730 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:38.532939 30027 consensus_queue.cc:1048] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:38.461030 28540 raft_consensus.cc:3060] T 9ead386562eb4fd787c7aee776c2194f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.482549 28546 raft_consensus.cc:3060] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.499239 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e25027f423144b088dad955c7282921" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:38.539068 28618 leader_election.cc:343] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:38.539481 28617 leader_election.cc:343] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 5e25027f423144b088dad955c7282921
I20260618 13:59:38.539569 28617 leader_election.cc:304] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:38.540482 28797 tablet_bootstrap.cc:492] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:38.540602 28797 ts_tablet_manager.cc:1403] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.033s user 0.005s sys 0.000s
I20260618 13:59:38.541148 30027 raft_consensus.cc:2749] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.541298 29745 ts_tablet_manager.cc:1929] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.541564 28797 raft_consensus.cc:359] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.541718 28797 raft_consensus.cc:385] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.541764 28797 raft_consensus.cc:740] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.541980 28797 consensus_queue.cc:260] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.542328 29745 log.cc:1199] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/0463fa2e30554abcbfc8f37b702e81d4
I20260618 13:59:38.542568 28797 ts_tablet_manager.cc:1434] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:38.543040 29745 ts_tablet_manager.cc:1950] T 0463fa2e30554abcbfc8f37b702e81d4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
W20260618 13:59:38.546537 28988 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 56650c9e20e24a28821cca4279bfde58 consensus state has no leader
I20260618 13:59:38.546380 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 103) took 5556 ms. Trace:
I20260618 13:59:38.546706 29745 rpcz_store.cc:276] 0618 13:59:32.990196 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.990238 (+ 42us) service_pool.cc:224] Handling call
0618 13:59:38.511121 (+5520883us) ts_tablet_manager.cc:1151] Deleting tablet 0463fa2e30554abcbfc8f37b702e81d4
0618 13:59:38.511132 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.520520 (+ 9388us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.541230 (+ 20710us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.546352 (+ 5122us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5520722}
I20260618 13:59:38.521306 28798 tablet_bootstrap.cc:492] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:38.547119 29745 tablet_replica.cc:333] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.547281 28798 ts_tablet_manager.cc:1403] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.190s user 0.028s sys 0.031s
I20260618 13:59:38.547298 29745 raft_consensus.cc:2243] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.547881 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 0463fa2e30554abcbfc8f37b702e81d4 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]) successfully deleted
I20260618 13:59:38.524019 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f83bb8c3c504d5f9affbaa8be3f1508" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:38.548239 28798 raft_consensus.cc:359] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.525231 29759 raft_consensus.cc:493] T 4bc87770baa34842a25ed97a19354446 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.547457 29745 raft_consensus.cc:2272] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.548501 28798 raft_consensus.cc:385] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.548578 29759 raft_consensus.cc:515] T 4bc87770baa34842a25ed97a19354446 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:38.549134 29745 ts_tablet_manager.cc:1916] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.549348 29759 leader_election.cc:290] T 4bc87770baa34842a25ed97a19354446 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:38.550376 28618 leader_election.cc:343] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.550689 28549 raft_consensus.cc:3060] T 4bc87770baa34842a25ed97a19354446 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.554443 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bc87770baa34842a25ed97a19354446" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:38.556700 29746 log.cc:1199] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/8fd9fa9675234db2abff694a517c0ea8
I20260618 13:59:38.556988 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bc87770baa34842a25ed97a19354446" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:38.557225 29746 ts_tablet_manager.cc:1950] T 8fd9fa9675234db2abff694a517c0ea8 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:38.557623 28484 leader_election.cc:304] T 4bc87770baa34842a25ed97a19354446 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.558087 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 108) took 3916 ms. Trace:
I20260618 13:59:38.558297 30151 raft_consensus.cc:493] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.558229 29746 rpcz_store.cc:276] 0618 13:59:34.641729 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.646198 (+ 4469us) service_pool.cc:224] Handling call
0618 13:59:38.237881 (+3591683us) ts_tablet_manager.cc:1151] Deleting tablet 8fd9fa9675234db2abff694a517c0ea8
0618 13:59:38.237895 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.357998 (+120103us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.442221 (+ 84223us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.558026 (+115805us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3591534}
I20260618 13:59:38.558485 28797 tablet_bootstrap.cc:492] T 054a7afb3b644aa49df78302cf8f02c9 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:38.558449 30151 raft_consensus.cc:515] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.558878 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 8fd9fa9675234db2abff694a517c0ea8 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]) successfully deleted
I20260618 13:59:38.559019 30151 leader_election.cc:290] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:38.559307 29746 tablet_replica.cc:333] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:38.559576 29746 raft_consensus.cc:2243] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.559832 29746 raft_consensus.cc:2272] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.560045 28797 tablet_bootstrap.cc:654] T 054a7afb3b644aa49df78302cf8f02c9 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.560132 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fea86ca38a741238a59053151087f69" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:38.560685 29746 ts_tablet_manager.cc:1916] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:38.560722 28346 leader_election.cc:343] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.561126 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fea86ca38a741238a59053151087f69" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:38.561288 30027 raft_consensus.cc:493] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.561393 30027 raft_consensus.cc:515] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:38.561672 28346 leader_election.cc:343] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.561764 28346 leader_election.cc:304] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.561940 30027 leader_election.cc:290] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.562168 30151 raft_consensus.cc:2749] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.548578 28798 raft_consensus.cc:740] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.562537 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba53ad1988064049a61fcca3a85f8779" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:38.562763 29745 ts_tablet_manager.cc:1929] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.562722 28798 consensus_queue.cc:260] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:38.563055 28618 leader_election.cc:343] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.563676 29745 log.cc:1199] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/6eea4d59935d40aa97dc10ec7469aa56
I20260618 13:59:38.564154 29745 ts_tablet_manager.cc:1950] T 6eea4d59935d40aa97dc10ec7469aa56 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.564160 28798 ts_tablet_manager.cc:1434] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.017s user 0.001s sys 0.001s
I20260618 13:59:38.565244 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 104) took 5574 ms. Trace:
I20260618 13:59:38.566269 29745 rpcz_store.cc:276] 0618 13:59:32.990581 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.990625 (+ 44us) service_pool.cc:224] Handling call
0618 13:59:38.547060 (+5556435us) ts_tablet_manager.cc:1151] Deleting tablet 6eea4d59935d40aa97dc10ec7469aa56
0618 13:59:38.547074 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.555951 (+ 8877us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.562707 (+ 6756us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.565226 (+ 2519us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5556288}
I20260618 13:59:38.567157 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 6eea4d59935d40aa97dc10ec7469aa56 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]) successfully deleted
I20260618 13:59:38.567232 29745 tablet_replica.cc:333] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.567433 29745 raft_consensus.cc:2243] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.564440 28798 tablet_bootstrap.cc:492] T 41c92a594cd64982a8972e4a019d1cd8 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:38.567945 30027 consensus_queue.cc:1048] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:38.568059 29745 raft_consensus.cc:2272] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.568656 29745 ts_tablet_manager.cc:1916] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.568874 28548 raft_consensus.cc:3060] T d464dd402d2044469e364ed37fdfb39b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.575017 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f83bb8c3c504d5f9affbaa8be3f1508" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:38.575170 28403 raft_consensus.cc:3060] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.577472 28403 raft_consensus.cc:2468] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:38.579530 29745 ts_tablet_manager.cc:1929] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:38.581301 29745 log.cc:1199] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/55ae29c3b7ed44cfa659130745037aa6
I20260618 13:59:38.581898 29745 ts_tablet_manager.cc:1950] T 55ae29c3b7ed44cfa659130745037aa6 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.583079 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 105) took 5592 ms. Trace:
I20260618 13:59:38.584033 29745 rpcz_store.cc:276] 0618 13:59:32.990933 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.990976 (+ 43us) service_pool.cc:224] Handling call
0618 13:59:38.567182 (+5576206us) ts_tablet_manager.cc:1151] Deleting tablet 55ae29c3b7ed44cfa659130745037aa6
0618 13:59:38.567193 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.572352 (+ 5159us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.579461 (+ 7109us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.583059 (+ 3598us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5576070}
I20260618 13:59:38.584721 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 55ae29c3b7ed44cfa659130745037aa6 (table default.Table_5d485acde41448d984aa79fa58b1866b [id=726324d365b94ff6bac24030222ca34f]) successfully deleted
I20260618 13:59:38.584818 29745 tablet_replica.cc:333] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.585067 29745 raft_consensus.cc:2243] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.507921 29025 catalog_manager.cc:5696] T 9ab1ddf79b094ba2aeee9225c0b332c3 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:38.585728 29745 raft_consensus.cc:2272] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.585762 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:38.586030 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:38.586314 29025 catalog_manager.cc:5696] T f54910015987484e95b295352d859315 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } }
I20260618 13:59:38.586551 29745 ts_tablet_manager.cc:1916] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.586828 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:38.587136 29025 catalog_manager.cc:5696] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:38.587697 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:38.588049 29025 catalog_manager.cc:5696] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:38.588661 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:38.596812 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba53ad1988064049a61fcca3a85f8779" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:38.597018 28411 raft_consensus.cc:2468] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:38.598536 29745 ts_tablet_manager.cc:1929] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.600098 29745 log.cc:1199] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/02b5cc08ca0c4e719e0414b345211ba5
I20260618 13:59:38.600626 29745 ts_tablet_manager.cc:1950] T 02b5cc08ca0c4e719e0414b345211ba5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.601738 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 106) took 5610 ms. Trace:
I20260618 13:59:38.602975 29745 rpcz_store.cc:276] 0618 13:59:33.001071 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:33.001125 (+ 54us) service_pool.cc:224] Handling call
0618 13:59:38.584769 (+5583644us) ts_tablet_manager.cc:1151] Deleting tablet 02b5cc08ca0c4e719e0414b345211ba5
0618 13:59:38.584779 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.590898 (+ 6119us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.598487 (+ 7589us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.601719 (+ 3232us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5583437}
I20260618 13:59:38.603787 29745 tablet_replica.cc:333] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.603644 29894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 623 } } ignore_live_leader: false dest_uuid: "7637285570574a6dbcef1de6f84ecac0" is_pre_election: true
I20260618 13:59:38.604245 29745 raft_consensus.cc:2243] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.604393 29745 raft_consensus.cc:2272] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.604928 29745 ts_tablet_manager.cc:1916] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.604795 28071 leader_election.cc:304] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e4d4f4566be42dfbafb7c0e40fb17ff; no voters: 7637285570574a6dbcef1de6f84ecac0, a950ab7cd66b4cb0b29592aa692c2dd0
I20260618 13:59:38.605311 30340 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260618 13:59:38.608705 29749 raft_consensus.cc:493] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.609005 28798 tablet_bootstrap.cc:654] T 41c92a594cd64982a8972e4a019d1cd8 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.608914 29749 raft_consensus.cc:515] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.609917 29749 leader_election.cc:290] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.610206 30151 raft_consensus.cc:493] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.610348 30151 raft_consensus.cc:515] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.610486 30338 raft_consensus.cc:493] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.610693 30338 raft_consensus.cc:515] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:38.610816 28484 leader_election.cc:343] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 18d8b609b96a4e81b57f69bff522eeea
I20260618 13:59:38.610966 30151 leader_election.cc:290] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:38.611281 30338 leader_election.cc:290] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:38.611653 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db62b3a1e37b447c9ea3fbd65c0cf1e7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:38.612178 28346 leader_election.cc:343] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:38.612705 28618 consensus_peers.cc:597] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.613245 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab521028ac1a4d0da013112d76bb169f" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:38.613525 29400 consensus_queue.cc:1048] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:38.614521 28617 consensus_peers.cc:597] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 015494374645469785f1f978eaf0e9a5. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.614549 29745 ts_tablet_manager.cc:1929] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.614719 29745 log.cc:1199] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/e6de1293572f4e75af3417f281b9a142
I20260618 13:59:38.615149 29745 ts_tablet_manager.cc:1950] T e6de1293572f4e75af3417f281b9a142 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.616029 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 107) took 5602 ms. Trace:
I20260618 13:59:38.616135 29745 rpcz_store.cc:276] 0618 13:59:33.014059 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:33.014096 (+ 37us) service_pool.cc:224] Handling call
0618 13:59:38.603741 (+5589645us) ts_tablet_manager.cc:1151] Deleting tablet e6de1293572f4e75af3417f281b9a142
0618 13:59:38.603750 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.611623 (+ 7873us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.614493 (+ 2870us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.616007 (+ 1514us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5589477}
I20260618 13:59:38.616530 29745 tablet_replica.cc:333] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.616775 29745 raft_consensus.cc:2243] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.616951 29745 raft_consensus.cc:2272] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.617547 29745 ts_tablet_manager.cc:1916] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:38.618127 28483 leader_election.cc:343] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 18d8b609b96a4e81b57f69bff522eeea
I20260618 13:59:38.618263 28483 leader_election.cc:304] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:38.619745 28483 leader_election.cc:343] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: ab521028ac1a4d0da013112d76bb169f
I20260618 13:59:38.619890 28483 leader_election.cc:304] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.620339 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db62b3a1e37b447c9ea3fbd65c0cf1e7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:38.620718 30338 raft_consensus.cc:2749] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:38.621260 28346 leader_election.cc:343] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.621390 28346 leader_election.cc:304] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.621445 30338 raft_consensus.cc:2749] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.621590 30151 raft_consensus.cc:2749] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.622463 28799 tablet_bootstrap.cc:492] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:38.622490 29759 raft_consensus.cc:2749] T 4bc87770baa34842a25ed97a19354446 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.622588 28799 ts_tablet_manager.cc:1403] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.134s user 0.016s sys 0.037s
I20260618 13:59:38.623157 28799 raft_consensus.cc:359] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.623298 28799 raft_consensus.cc:385] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.623350 28799 raft_consensus.cc:740] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.623521 28799 consensus_queue.cc:260] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.623946 28799 ts_tablet_manager.cc:1434] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 13:59:38.624178 28799 tablet_bootstrap.cc:492] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:38.625437 28799 tablet_bootstrap.cc:654] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.631294 29745 ts_tablet_manager.cc:1929] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.631405 29745 log.cc:1199] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/827cc0987bd541759e334fb620f04ed5
I20260618 13:59:38.631747 29745 ts_tablet_manager.cc:1950] T 827cc0987bd541759e334fb620f04ed5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.632447 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 109) took 5618 ms. Trace:
I20260618 13:59:38.632534 29745 rpcz_store.cc:276] 0618 13:59:33.014237 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:33.014314 (+ 77us) service_pool.cc:224] Handling call
0618 13:59:38.616476 (+5602162us) ts_tablet_manager.cc:1151] Deleting tablet 827cc0987bd541759e334fb620f04ed5
0618 13:59:38.616488 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.628620 (+ 12132us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.631251 (+ 2631us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.632430 (+ 1179us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5601968}
I20260618 13:59:38.639019 29745 tablet_replica.cc:333] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.639225 29745 raft_consensus.cc:2243] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.639382 29745 raft_consensus.cc:2272] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.641405 29745 ts_tablet_manager.cc:1916] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.617096 28797 tablet_bootstrap.cc:492] T 054a7afb3b644aa49df78302cf8f02c9 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:38.643497 28797 ts_tablet_manager.cc:1403] T 054a7afb3b644aa49df78302cf8f02c9 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.085s user 0.003s sys 0.000s
I20260618 13:59:38.644248 28797 raft_consensus.cc:359] T 054a7afb3b644aa49df78302cf8f02c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.644399 28797 raft_consensus.cc:385] T 054a7afb3b644aa49df78302cf8f02c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.644451 28797 raft_consensus.cc:740] T 054a7afb3b644aa49df78302cf8f02c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.644611 28797 consensus_queue.cc:260] T 054a7afb3b644aa49df78302cf8f02c9 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.645073 28797 ts_tablet_manager.cc:1434] T 054a7afb3b644aa49df78302cf8f02c9 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 13:59:38.645215 28799 tablet_bootstrap.cc:492] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:38.645316 28797 tablet_bootstrap.cc:492] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:38.645318 28799 ts_tablet_manager.cc:1403] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.021s user 0.010s sys 0.000s
I20260618 13:59:38.645931 28799 raft_consensus.cc:359] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.646066 28799 raft_consensus.cc:385] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.646114 28799 raft_consensus.cc:740] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.646319 28799 consensus_queue.cc:260] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:38.619527 28618 consensus_peers.cc:597] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.646888 28799 ts_tablet_manager.cc:1434] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:38.647127 28799 tablet_bootstrap.cc:492] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:38.648452 28799 tablet_bootstrap.cc:654] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.649294 28797 tablet_bootstrap.cc:654] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.603752 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 02b5cc08ca0c4e719e0414b345211ba5 (table default.Table_6f3c7c1d308e4901887fc9b602b38e4f [id=7aa3803c01134af6b073952eb9843282]) successfully deleted
W20260618 13:59:38.650765 28988 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 47044.4us
I20260618 13:59:38.651018 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet e6de1293572f4e75af3417f281b9a142 (table default.Table_f75651a1e55f4271b39388713dcd74aa [id=6705bb69561d435496d424c0d64f9879]) successfully deleted
I20260618 13:59:38.651188 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 827cc0987bd541759e334fb620f04ed5 (table default.Table_9bfb035b965742abaaf21413be078bf6 [id=9be95b9adc544c0e9675071330a21225]) successfully deleted
I20260618 13:59:38.651793 29025 catalog_manager.cc:5696] T d464dd402d2044469e364ed37fdfb39b P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:38.652443 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:38.652755 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:38.653676 29745 ts_tablet_manager.cc:1929] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.653780 29745 log.cc:1199] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/bc0adedecee4405e86b0a37b9e62e352
I20260618 13:59:38.653805 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:38.654122 29745 ts_tablet_manager.cc:1950] T bc0adedecee4405e86b0a37b9e62e352 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.654857 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 110) took 5640 ms. Trace:
I20260618 13:59:38.654742 29025 catalog_manager.cc:5696] T 8ed88c142d7d40cda6392c06bc0486ca P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:38.654942 29745 rpcz_store.cc:276] 0618 13:59:33.014796 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:33.014844 (+ 48us) service_pool.cc:224] Handling call
0618 13:59:38.638970 (+5624126us) ts_tablet_manager.cc:1151] Deleting tablet bc0adedecee4405e86b0a37b9e62e352
0618 13:59:38.638980 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.649840 (+ 10860us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.653630 (+ 3790us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.654838 (+ 1208us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5623979}
W20260618 13:59:38.656757 28618 consensus_peers.cc:597] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.657480 29745 tablet_replica.cc:333] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.657683 29745 raft_consensus.cc:2243] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.657821 29745 raft_consensus.cc:2272] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.658331 29745 ts_tablet_manager.cc:1916] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.643513 28617 leader_election.cc:304] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:38.658834 30027 raft_consensus.cc:2804] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:38.658963 30027 raft_consensus.cc:697] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:38.659206 30027 consensus_queue.cc:237] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.662849 28797 tablet_bootstrap.cc:492] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:38.663666 28797 ts_tablet_manager.cc:1403] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.000s
I20260618 13:59:38.664587 28797 raft_consensus.cc:359] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.664951 29746 ts_tablet_manager.cc:1929] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.665655 28797 raft_consensus.cc:385] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.665763 28797 raft_consensus.cc:740] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.666102 29745 ts_tablet_manager.cc:1929] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.666204 29745 log.cc:1199] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/cd06bb7cffc34412b420d70435fce373
I20260618 13:59:38.666283 28797 consensus_queue.cc:260] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.666553 29745 ts_tablet_manager.cc:1950] T cd06bb7cffc34412b420d70435fce373 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.667232 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 111) took 5652 ms. Trace:
I20260618 13:59:38.667387 28797 ts_tablet_manager.cc:1434] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260618 13:59:38.667313 29745 rpcz_store.cc:276] 0618 13:59:33.015116 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:33.015161 (+ 45us) service_pool.cc:224] Handling call
0618 13:59:38.657431 (+5642270us) ts_tablet_manager.cc:1151] Deleting tablet cd06bb7cffc34412b420d70435fce373
0618 13:59:38.657442 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.663378 (+ 5936us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.666059 (+ 2681us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.667215 (+ 1156us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5641677}
I20260618 13:59:38.667701 28797 tablet_bootstrap.cc:492] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:38.669693 28797 tablet_bootstrap.cc:654] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.674193 29745 tablet_replica.cc:333] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.674440 29745 raft_consensus.cc:2243] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:38.674729 29745 raft_consensus.cc:2272] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.675209 29745 ts_tablet_manager.cc:1916] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.677486 28617 leader_election.cc:304] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:38.678320 29303 raft_consensus.cc:2804] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:38.678393 29303 raft_consensus.cc:493] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:38.678476 29303 raft_consensus.cc:3060] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.681058 29303 raft_consensus.cc:515] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.681654 29745 ts_tablet_manager.cc:1929] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.681744 29745 log.cc:1199] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/591465e2b57247a2b224cc8b09225811
I20260618 13:59:38.682087 29745 ts_tablet_manager.cc:1950] T 591465e2b57247a2b224cc8b09225811 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.682298 29303 leader_election.cc:290] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.631968 29566 ts_tablet_manager.cc:1929] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.683279 29566 log.cc:1199] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/c557afc3abfa4ef188bf248e89f49d87
I20260618 13:59:38.687062 29566 ts_tablet_manager.cc:1950] T c557afc3abfa4ef188bf248e89f49d87 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:38.687587 29303 consensus_queue.cc:1048] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:38.689946 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba53ad1988064049a61fcca3a85f8779" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:38.690150 28408 raft_consensus.cc:3060] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.690593 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 89) took 6578 ms. Trace:
I20260618 13:59:38.690667 29566 rpcz_store.cc:276] 0618 13:59:32.112248 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.112304 (+ 56us) service_pool.cc:224] Handling call
0618 13:59:38.496195 (+6383891us) ts_tablet_manager.cc:1151] Deleting tablet c557afc3abfa4ef188bf248e89f49d87
0618 13:59:38.496204 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.536150 (+ 39946us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.631919 (+ 95769us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.690576 (+ 58657us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6383692}
I20260618 13:59:38.690986 29566 tablet_replica.cc:333] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:38.691170 29566 raft_consensus.cc:2243] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:38.691453 29566 raft_consensus.cc:2272] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.692066 29566 ts_tablet_manager.cc:1916] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.693459 28408 raft_consensus.cc:2468] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:38.704507 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 108) took 5690 ms. Trace:
I20260618 13:59:38.704612 29745 rpcz_store.cc:276] 0618 13:59:33.014144 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:33.016712 (+ 2568us) service_pool.cc:224] Handling call
0618 13:59:38.674138 (+5657426us) ts_tablet_manager.cc:1151] Deleting tablet 591465e2b57247a2b224cc8b09225811
0618 13:59:38.674149 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.678374 (+ 4225us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.681612 (+ 3238us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.704484 (+ 22872us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5657087}
W20260618 13:59:38.705273 28618 consensus_peers.cc:597] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.705516 29745 tablet_replica.cc:333] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.705711 29745 raft_consensus.cc:2243] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.705889 29745 raft_consensus.cc:2272] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.706403 29745 ts_tablet_manager.cc:1916] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.655383 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet bc0adedecee4405e86b0a37b9e62e352 (table default.Table_cc20b3f971bb46969baf1fc71e217f2c [id=0a955d9a7fec4cd7985ac56e210fe8a6]) successfully deleted
I20260618 13:59:38.691455 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet c557afc3abfa4ef188bf248e89f49d87 (table default.Table_fdff0afd87584a53a884c9b76926d6d0 [id=7cd5262564144438b00bde60a6cee1e6]) successfully deleted
W20260618 13:59:38.710265 28988 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 54932.5us
I20260618 13:59:38.710624 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet cd06bb7cffc34412b420d70435fce373 (table default.Table_74fc7c21dc664ff8bfe0d25aee399bdb [id=d224150fd3a340fc96e75c0b96146b11]) successfully deleted
I20260618 13:59:38.710839 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 591465e2b57247a2b224cc8b09225811 (table default.Table_0a85d79af65f48a3803bfd1248593a11 [id=2f4d62be772a421ab777fd23be58b719]) successfully deleted
I20260618 13:59:38.711995 29025 catalog_manager.cc:5696] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:38.712508 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:38.712749 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:38.713315 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:38.713570 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:38.714012 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:38.714767 29025 catalog_manager.cc:5696] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:38.715984 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:38.716399 29025 catalog_manager.cc:5696] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:38.718149 29745 ts_tablet_manager.cc:1929] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.718266 29745 log.cc:1199] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/46bfe8cc2078414d84e65e22ed4ef8a2
I20260618 13:59:38.718619 29745 ts_tablet_manager.cc:1950] T 46bfe8cc2078414d84e65e22ed4ef8a2 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.721174 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 124) took 5080 ms. Trace:
I20260618 13:59:38.721262 29745 rpcz_store.cc:276] 0618 13:59:33.641201 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:33.641261 (+ 60us) service_pool.cc:224] Handling call
0618 13:59:38.705467 (+5064206us) ts_tablet_manager.cc:1151] Deleting tablet 46bfe8cc2078414d84e65e22ed4ef8a2
0618 13:59:38.705478 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.712160 (+ 6682us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.718096 (+ 5936us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.721154 (+ 3058us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5064013}
I20260618 13:59:38.722101 29745 tablet_replica.cc:333] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.722304 29745 raft_consensus.cc:2243] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.722945 29745 raft_consensus.cc:2272] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.723457 29745 ts_tablet_manager.cc:1916] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.731683 29745 ts_tablet_manager.cc:1929] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.731784 29745 log.cc:1199] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/9a0691de253b4b008e5d995fd6b876c2
I20260618 13:59:38.682299 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba53ad1988064049a61fcca3a85f8779" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:38.731971 28797 tablet_bootstrap.cc:492] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:38.695312 29749 raft_consensus.cc:493] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:38.732136 29745 ts_tablet_manager.cc:1950] T 9a0691de253b4b008e5d995fd6b876c2 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.732134 28797 ts_tablet_manager.cc:1403] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.065s user 0.005s sys 0.000s
I20260618 13:59:38.732095 29749 raft_consensus.cc:515] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.732707 29749 leader_election.cc:290] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.732775 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 125) took 4808 ms. Trace:
I20260618 13:59:38.732860 29745 rpcz_store.cc:276] 0618 13:59:33.924265 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:33.924329 (+ 64us) service_pool.cc:224] Handling call
0618 13:59:38.722057 (+4797728us) ts_tablet_manager.cc:1151] Deleting tablet 9a0691de253b4b008e5d995fd6b876c2
0618 13:59:38.722067 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.729142 (+ 7075us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.731639 (+ 2497us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.732758 (+ 1119us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4797460}
I20260618 13:59:38.733060 28797 raft_consensus.cc:359] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.733258 28797 raft_consensus.cc:385] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.733320 28797 raft_consensus.cc:740] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.708351 28798 tablet_bootstrap.cc:492] T 41c92a594cd64982a8972e4a019d1cd8 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
W20260618 13:59:38.733546 28618 leader_election.cc:343] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:38.728467 28483 consensus_peers.cc:597] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4cfca9e76e641a7bc0b58b9ee240f36. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.728616 30338 raft_consensus.cc:493] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:38.734109 30338 raft_consensus.cc:515] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:38.734661 28484 leader_election.cc:343] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 751cd9bd961b4f56a588ed02a05f0212
I20260618 13:59:38.734714 30338 leader_election.cc:290] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.735229 29746 log.cc:1199] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/41107eb1d9164a9c98060003b4027412
W20260618 13:59:38.735659 28484 leader_election.cc:343] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: d667ea5f029f46d89fcb3be03069b97b
W20260618 13:59:38.735698 28483 leader_election.cc:343] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 751cd9bd961b4f56a588ed02a05f0212
I20260618 13:59:38.735809 28483 leader_election.cc:304] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.736191 29746 ts_tablet_manager.cc:1950] T 41107eb1d9164a9c98060003b4027412 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:38.736533 30338 raft_consensus.cc:2749] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:38.736897 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 118) took 4090 ms. Trace:
W20260618 13:59:38.736980 28483 leader_election.cc:343] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: d667ea5f029f46d89fcb3be03069b97b
I20260618 13:59:38.737100 28483 leader_election.cc:304] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.737493 30338 raft_consensus.cc:2749] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:38.737808 28798 ts_tablet_manager.cc:1403] T 41c92a594cd64982a8972e4a019d1cd8 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.173s user 0.001s sys 0.005s
I20260618 13:59:38.738649 28798 raft_consensus.cc:359] T 41c92a594cd64982a8972e4a019d1cd8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.738859 28798 raft_consensus.cc:385] T 41c92a594cd64982a8972e4a019d1cd8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.738976 28798 raft_consensus.cc:740] T 41c92a594cd64982a8972e4a019d1cd8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.739234 28798 consensus_queue.cc:260] T 41c92a594cd64982a8972e4a019d1cd8 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.739856 28798 ts_tablet_manager.cc:1434] T 41c92a594cd64982a8972e4a019d1cd8 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:38.736999 29746 rpcz_store.cc:276] 0618 13:59:34.646159 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.646379 (+ 220us) service_pool.cc:224] Handling call
0618 13:59:38.559207 (+3912828us) ts_tablet_manager.cc:1151] Deleting tablet 41107eb1d9164a9c98060003b4027412
0618 13:59:38.559221 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.662043 (+102822us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.664874 (+ 2831us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.736879 (+ 72005us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3912670}
I20260618 13:59:38.740769 28798 tablet_bootstrap.cc:492] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:38.742194 28798 tablet_bootstrap.cc:654] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:38.744793 28617 consensus_peers.cc:597] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2f3d2f236fd44ab8af0524ea282cfa94. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.745148 28617 leader_election.cc:304] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:38.721683 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 46bfe8cc2078414d84e65e22ed4ef8a2 (table default.Table_659a3d495ed44be4bfb2193816362e25 [id=2659803e822d4761a15e0e89644cc472]) successfully deleted
I20260618 13:59:38.741886 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 41107eb1d9164a9c98060003b4027412 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]) successfully deleted
I20260618 13:59:38.748817 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 9a0691de253b4b008e5d995fd6b876c2 (table default.Table_ab57b782a92747e78e2bfe28480aa0ec [id=28d43fd5660f459a9acfab2fc0dfbdb1]) successfully deleted
I20260618 13:59:38.749357 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:38.749866 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 2128dbec33db453ba51070767a15b21e (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]). Expected version 2 got 0
W20260618 13:59:38.750770 28617 consensus_peers.cc:597] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 04dac931e62a448d99633878eb70b92f. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.750934 29714 consensus_queue.cc:1048] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:38.751282 30274 raft_consensus.cc:2804] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:38.751401 30274 raft_consensus.cc:697] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:38.751616 30274 consensus_queue.cc:237] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:38.752130 28618 consensus_peers.cc:597] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.754709 29746 tablet_replica.cc:333] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:38.754963 29746 raft_consensus.cc:2243] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.755825 29746 raft_consensus.cc:2272] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.756388 29746 ts_tablet_manager.cc:1916] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.758765 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:38.759948 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:38.760840 30151 raft_consensus.cc:493] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.760962 30151 raft_consensus.cc:515] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.761003 29025 catalog_manager.cc:5696] T ad045effb32948b5b8e9be5bb7996577 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:38.761636 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:38.760773 29400 raft_consensus.cc:493] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.762089 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3d8616846c649b6a969b971db0d03db" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:38.762238 29400 raft_consensus.cc:515] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.762778 29400 leader_election.cc:290] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.763375 29759 consensus_queue.cc:1048] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:38.763891 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e99a85e83c46afa367e59e5ddb7dc7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:38.764422 28618 leader_election.cc:343] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:38.765030 28346 leader_election.cc:343] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.765456 30151 leader_election.cc:290] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:38.766014 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3d8616846c649b6a969b971db0d03db" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:38.766541 28346 leader_election.cc:343] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.766649 28346 leader_election.cc:304] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.766959 30152 raft_consensus.cc:2749] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.768468 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e99a85e83c46afa367e59e5ddb7dc7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:38.768675 28402 raft_consensus.cc:2468] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:38.773088 28797 consensus_queue.cc:260] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.773640 28797 ts_tablet_manager.cc:1434] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.041s user 0.001s sys 0.000s
I20260618 13:59:38.773890 28797 tablet_bootstrap.cc:492] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:38.774760 28617 consensus_peers.cc:597] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f3ba9186b3ff47d5a2b158fef619686e. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.774886 28617 leader_election.cc:304] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:38.775949 28798 tablet_bootstrap.cc:492] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:38.776070 28798 ts_tablet_manager.cc:1403] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.000s
I20260618 13:59:38.776777 28798 raft_consensus.cc:359] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.776928 28798 raft_consensus.cc:385] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.776973 28798 raft_consensus.cc:740] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.777149 28798 consensus_queue.cc:260] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.777588 28798 ts_tablet_manager.cc:1434] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 13:59:38.777846 28798 tablet_bootstrap.cc:492] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:38.779726 28797 tablet_bootstrap.cc:654] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.781934 28458 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:38.786227 30274 consensus_queue.cc:1048] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:38.786584 30274 raft_consensus.cc:2804] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:38.786682 30274 raft_consensus.cc:493] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:38.787005 30274 raft_consensus.cc:3060] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:38.787034 28618 consensus_peers.cc:597] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.791352 29566 ts_tablet_manager.cc:1929] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.791476 29566 log.cc:1199] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/4c2b924a096b479cad31c29bf7c99172
I20260618 13:59:38.791880 29566 ts_tablet_manager.cc:1950] T 4c2b924a096b479cad31c29bf7c99172 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:38.792604 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 90) took 6679 ms. Trace:
I20260618 13:59:38.792693 29566 rpcz_store.cc:276] 0618 13:59:32.113559 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.113613 (+ 54us) service_pool.cc:224] Handling call
0618 13:59:38.690941 (+6577328us) ts_tablet_manager.cc:1151] Deleting tablet 4c2b924a096b479cad31c29bf7c99172
0618 13:59:38.690952 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.728724 (+ 37772us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.791298 (+ 62574us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.792587 (+ 1289us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6577133}
I20260618 13:59:38.793036 29566 tablet_replica.cc:333] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:38.793224 29566 raft_consensus.cc:2243] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.793363 29566 raft_consensus.cc:2272] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.793895 29566 ts_tablet_manager.cc:1916] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.795251 28798 tablet_bootstrap.cc:654] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.795850 30274 raft_consensus.cc:515] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.797601 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e99a85e83c46afa367e59e5ddb7dc7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:38.797294 30274 leader_election.cc:290] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:38.798032 28618 leader_election.cc:343] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.802575 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e99a85e83c46afa367e59e5ddb7dc7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:38.802732 28408 raft_consensus.cc:3060] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.803180 30274 consensus_queue.cc:1048] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:38.793298 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 4c2b924a096b479cad31c29bf7c99172 (table default.Table_3deacb2c9668491a85bbb09ae3f57d66 [id=431a660bfbed46248a603dc7781cf543]) successfully deleted
I20260618 13:59:38.762082 29025 catalog_manager.cc:5696] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:38.807176 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:38.807451 28798 tablet_bootstrap.cc:492] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:38.807605 28798 ts_tablet_manager.cc:1403] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.000s
I20260618 13:59:38.807642 29025 catalog_manager.cc:5696] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:38.808172 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_c70d6c5b0dc94b33b4c5772edd2b9dca [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 2 got 0
I20260618 13:59:38.808404 28798 raft_consensus.cc:359] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.808754 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:38.809244 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 72c04f55890648f0a8184f3783243f7c (table default.Table_773be314d7e741fcb4fb6eb7d954802e [id=0d83fc68255c458587390c58b12e00d7]). Expected version 2 got 0
I20260618 13:59:38.809976 29025 catalog_manager.cc:5696] T 657e55fa53c447babfe91b1582806e6b P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:38.808568 28798 raft_consensus.cc:385] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.812079 28798 raft_consensus.cc:740] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.812685 28798 consensus_queue.cc:260] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.814874 29745 tablet_replica.cc:333] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.811785 29746 ts_tablet_manager.cc:1929] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:38.815148 29746 log.cc:1199] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/fca76774f119435899188313a33dad98
I20260618 13:59:38.815202 29745 raft_consensus.cc:2243] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.815497 29745 raft_consensus.cc:2272] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.816159 29745 ts_tablet_manager.cc:1916] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.816973 29746 ts_tablet_manager.cc:1950] T fca76774f119435899188313a33dad98 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:38.825325 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 119) took 4178 ms. Trace:
I20260618 13:59:38.826412 29746 rpcz_store.cc:276] 0618 13:59:34.646697 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.646821 (+ 124us) service_pool.cc:224] Handling call
0618 13:59:38.740576 (+4093755us) ts_tablet_manager.cc:1151] Deleting tablet fca76774f119435899188313a33dad98
0618 13:59:38.754663 (+ 14087us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.760608 (+ 5945us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.811729 (+ 51121us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.825299 (+ 13570us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4093571}
I20260618 13:59:38.831010 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet fca76774f119435899188313a33dad98 (table default.Table_4cf88aaddada4361837d7d49170f029b [id=50525d6257fa4de8a00dbee44d37f24d]) successfully deleted
I20260618 13:59:38.832876 28408 raft_consensus.cc:2468] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:38.833297 28617 leader_election.cc:304] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:38.833731 30059 raft_consensus.cc:2804] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:38.833845 30059 raft_consensus.cc:697] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:38.834065 30059 consensus_queue.cc:237] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.817432 28798 ts_tablet_manager.cc:1434] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20260618 13:59:38.836649 28798 tablet_bootstrap.cc:492] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:38.838157 28798 tablet_bootstrap.cc:654] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.820398 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:38.840749 29025 catalog_manager.cc:5696] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } }
W20260618 13:59:38.846160 28618 consensus_peers.cc:597] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.839330 29746 tablet_replica.cc:333] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:38.847434 28549 raft_consensus.cc:3060] T ab100ebbe61a4321b9e76c730e5d46ed P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.850685 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a]). Expected version 1 got 0
I20260618 13:59:38.851015 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:38.851437 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:38.851820 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:38.852355 28799 tablet_bootstrap.cc:492] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:38.852492 28799 ts_tablet_manager.cc:1403] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.205s user 0.066s sys 0.040s
I20260618 13:59:38.852258 29025 catalog_manager.cc:5696] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:38.852847 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:38.853246 28799 raft_consensus.cc:359] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.853420 28799 raft_consensus.cc:385] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.853487 28799 raft_consensus.cc:740] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.853295 29025 catalog_manager.cc:5696] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:38.853973 29025 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:38.856496 29745 ts_tablet_manager.cc:1929] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:38.856596 29745 log.cc:1199] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/98b42d23832545ae81a7cdff19e4689f
I20260618 13:59:38.856928 29745 ts_tablet_manager.cc:1950] T 98b42d23832545ae81a7cdff19e4689f P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.857208 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 126) took 4877 ms. Trace:
I20260618 13:59:38.857285 29745 rpcz_store.cc:276] 0618 13:59:33.979369 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:33.979416 (+ 47us) service_pool.cc:224] Handling call
0618 13:59:38.814805 (+4835389us) ts_tablet_manager.cc:1151] Deleting tablet 98b42d23832545ae81a7cdff19e4689f
0618 13:59:38.814819 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.842514 (+ 27695us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.856445 (+ 13931us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.857195 (+ 750us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4835194}
I20260618 13:59:38.858412 28799 consensus_queue.cc:260] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.858929 28799 ts_tablet_manager.cc:1434] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260618 13:59:38.859187 28799 tablet_bootstrap.cc:492] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:38.860539 28799 tablet_bootstrap.cc:654] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.866361 29745 tablet_replica.cc:333] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.866322 30338 raft_consensus.cc:493] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.866508 30338 raft_consensus.cc:515] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.866600 29745 raft_consensus.cc:2243] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.866739 29745 raft_consensus.cc:2272] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.867218 29745 ts_tablet_manager.cc:1916] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.867205 30338 leader_election.cc:290] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:38.868754 29746 raft_consensus.cc:2243] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.868925 29746 raft_consensus.cc:2272] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.869521 29746 ts_tablet_manager.cc:1916] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.870157 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d8dd2c388e43a0be380aa953ec2c82" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:38.873519 28596 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:38.874727 29400 consensus_queue.cc:1048] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:38.878018 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d8dd2c388e43a0be380aa953ec2c82" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:38.878780 28483 leader_election.cc:304] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:38.879135 30338 raft_consensus.cc:2749] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:38.881315 29400 consensus_queue.cc:1048] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:38.857702 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 98b42d23832545ae81a7cdff19e4689f (table default.Table_c99584ecc88948a097b39c22d68b3938 [id=7c39b3c432e944bca05f7b2070a79ff9]) successfully deleted
W20260618 13:59:38.906980 28617 consensus_peers.cc:597] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ca2ee746c5024cfc9b31c470f9616e7d. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.907616 29745 ts_tablet_manager.cc:1929] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:38.907732 29745 log.cc:1199] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/962bf304a285412fb2a529db5b994e9e
I20260618 13:59:38.907773 30151 raft_consensus.cc:493] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.907935 30151 raft_consensus.cc:515] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.908062 29745 ts_tablet_manager.cc:1950] T 962bf304a285412fb2a529db5b994e9e P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.908634 30151 leader_election.cc:290] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:38.908808 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 127) took 4594 ms. Trace:
I20260618 13:59:38.908905 29745 rpcz_store.cc:276] 0618 13:59:34.313991 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.315064 (+ 1073us) service_pool.cc:224] Handling call
0618 13:59:38.866314 (+4551250us) ts_tablet_manager.cc:1151] Deleting tablet 962bf304a285412fb2a529db5b994e9e
0618 13:59:38.866325 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.870475 (+ 4150us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.907555 (+ 37080us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.908790 (+ 1235us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4550989}
I20260618 13:59:38.909211 29745 tablet_replica.cc:333] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.909354 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 962bf304a285412fb2a529db5b994e9e (table default.Table_0fbace1c1aa746a6a2a57c878631218b [id=18b5971926ed4fff9048003a51d3a4fa]) successfully deleted
I20260618 13:59:38.909401 29745 raft_consensus.cc:2243] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.909533 29745 raft_consensus.cc:2272] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.910032 29745 ts_tablet_manager.cc:1916] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.910110 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054a7afb3b644aa49df78302cf8f02c9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:38.910277 28680 raft_consensus.cc:2468] T 054a7afb3b644aa49df78302cf8f02c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:38.910702 28346 leader_election.cc:304] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:38.911038 30151 raft_consensus.cc:2804] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:38.911115 30151 raft_consensus.cc:493] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:38.911178 30151 raft_consensus.cc:3060] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:38.911983 28617 consensus_peers.cc:597] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a0691de253b4b008e5d995fd6b876c2. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:38.912204 28617 consensus_peers.cc:597] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bc0adedecee4405e86b0a37b9e62e352. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.912520 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054a7afb3b644aa49df78302cf8f02c9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:38.913061 28346 leader_election.cc:343] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.925087 29745 ts_tablet_manager.cc:1929] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:38.925206 29745 log.cc:1199] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/e3d8616846c649b6a969b971db0d03db
I20260618 13:59:38.925479 29745 ts_tablet_manager.cc:1950] T e3d8616846c649b6a969b971db0d03db P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.926784 28797 tablet_bootstrap.cc:492] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:38.926920 28797 ts_tablet_manager.cc:1403] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.153s user 0.008s sys 0.001s
I20260618 13:59:38.927647 28797 raft_consensus.cc:359] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:38.927817 28797 raft_consensus.cc:385] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.927862 28797 raft_consensus.cc:740] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.928081 28797 consensus_queue.cc:260] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:38.928602 28797 ts_tablet_manager.cc:1434] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:38.928593 30151 raft_consensus.cc:515] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.928853 28797 tablet_bootstrap.cc:492] T 340f6bbf18eb412ca3afec237ff7752d P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:38.929231 30151 leader_election.cc:290] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:38.930140 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054a7afb3b644aa49df78302cf8f02c9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:38.930143 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054a7afb3b644aa49df78302cf8f02c9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:38.930286 28680 raft_consensus.cc:3060] T 054a7afb3b644aa49df78302cf8f02c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:38.930661 28346 leader_election.cc:343] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:38.932675 28680 raft_consensus.cc:2468] T 054a7afb3b644aa49df78302cf8f02c9 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:38.933142 28346 leader_election.cc:304] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:38.933161 29566 ts_tablet_manager.cc:1929] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.933369 29566 log.cc:1199] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/da4d4f03911f4ac099ecc2fa1d8af494
I20260618 13:59:38.933507 30151 raft_consensus.cc:2804] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:38.933740 29566 ts_tablet_manager.cc:1950] T da4d4f03911f4ac099ecc2fa1d8af494 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:38.933722 30151 raft_consensus.cc:697] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:38.934005 30151 consensus_queue.cc:237] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.934465 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 91) took 6820 ms. Trace:
I20260618 13:59:38.934549 29566 rpcz_store.cc:276] 0618 13:59:32.114255 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.114297 (+ 42us) service_pool.cc:224] Handling call
0618 13:59:38.792989 (+6678692us) ts_tablet_manager.cc:1151] Deleting tablet da4d4f03911f4ac099ecc2fa1d8af494
0618 13:59:38.793000 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.845047 (+ 52047us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.933108 (+ 88061us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.934446 (+ 1338us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6678560}
I20260618 13:59:38.935089 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet da4d4f03911f4ac099ecc2fa1d8af494 (table default.Table_99428babdde649b8b5d5fab801b58136 [id=f7cb92d4ab0742af9946f6bcf708df4d]) successfully deleted
I20260618 13:59:38.936650 28549 raft_consensus.cc:3060] T 1d8b3cfac0f648cd8b1c51102132c676 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:38.937501 28617 consensus_peers.cc:597] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6eea4d59935d40aa97dc10ec7469aa56. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:38.944355 28618 consensus_peers.cc:597] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:38.944887 28798 tablet_bootstrap.cc:492] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:38.945086 28798 ts_tablet_manager.cc:1403] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.109s user 0.002s sys 0.001s
I20260618 13:59:38.946090 28798 raft_consensus.cc:359] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.946522 28798 raft_consensus.cc:385] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.946702 28798 raft_consensus.cc:740] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.947065 28798 consensus_queue.cc:260] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.951066 28797 tablet_bootstrap.cc:654] T 340f6bbf18eb412ca3afec237ff7752d P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.951538 28799 tablet_bootstrap.cc:492] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:38.951671 28799 ts_tablet_manager.cc:1403] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.093s user 0.007s sys 0.017s
I20260618 13:59:38.926333 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 128) took 4297 ms. Trace:
I20260618 13:59:38.951790 29745 rpcz_store.cc:276] 0618 13:59:34.628889 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.628958 (+ 69us) service_pool.cc:224] Handling call
0618 13:59:38.909165 (+4280207us) ts_tablet_manager.cc:1151] Deleting tablet e3d8616846c649b6a969b971db0d03db
0618 13:59:38.909175 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.922324 (+ 13149us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.925034 (+ 2710us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.926311 (+ 1277us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4280053}
I20260618 13:59:38.952256 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet e3d8616846c649b6a969b971db0d03db (table default.Table_473a0c23203b404ea8e5ee0aed201e7b [id=3656a2f8d6cc4892a071908ae55efd3f]) successfully deleted
I20260618 13:59:38.952400 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 129) took 4323 ms. Trace:
I20260618 13:59:38.952490 29745 rpcz_store.cc:276] 0618 13:59:34.629214 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.629247 (+ 33us) service_pool.cc:224] Handling call
0618 13:59:38.952248 (+4323001us) ts_tablet_manager.cc:1151] Deleting tablet f89c9954d48e407c96c294c26644e25c
0618 13:59:38.952257 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.952383 (+ 126us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4322881}
W20260618 13:59:38.952916 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet f89c9954d48e407c96c294c26644e25c because the tablet was not found. No further retry: Not found: Tablet not found: f89c9954d48e407c96c294c26644e25c
I20260618 13:59:38.953001 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 130) took 4323 ms. Trace:
I20260618 13:59:38.953066 29745 rpcz_store.cc:276] 0618 13:59:34.629461 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.629499 (+ 38us) service_pool.cc:224] Handling call
0618 13:59:38.952914 (+4323415us) ts_tablet_manager.cc:1151] Deleting tablet f278d0acbb0c4a0ca61640bc142e6335
0618 13:59:38.952923 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.952985 (+ 62us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4323303}
W20260618 13:59:38.953384 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet f278d0acbb0c4a0ca61640bc142e6335 because the tablet was not found. No further retry: Not found: Tablet not found: f278d0acbb0c4a0ca61640bc142e6335
I20260618 13:59:38.953436 29745 tablet_replica.cc:333] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:38.953634 29745 raft_consensus.cc:2243] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:38.953855 29745 raft_consensus.cc:2272] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.954097 28799 raft_consensus.cc:359] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.954249 28799 raft_consensus.cc:385] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.954294 28799 raft_consensus.cc:740] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.954639 29745 ts_tablet_manager.cc:1916] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.959626 28799 consensus_queue.cc:260] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.959755 28798 ts_tablet_manager.cc:1434] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.015s user 0.001s sys 0.001s
I20260618 13:59:38.960175 28798 tablet_bootstrap.cc:492] T f54910015987484e95b295352d859315 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:38.960706 30165 raft_consensus.cc:493] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:38.960822 30165 raft_consensus.cc:515] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.961428 30165 leader_election.cc:290] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:38.962510 28483 leader_election.cc:343] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 3dd0eaa64271489d92aa380df877f257
I20260618 13:59:38.962525 28799 ts_tablet_manager.cc:1434] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260618 13:59:38.962575 28798 tablet_bootstrap.cc:654] T f54910015987484e95b295352d859315 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.961952 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dd0eaa64271489d92aa380df877f257" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:38.966749 28680 raft_consensus.cc:2468] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:38.967232 28484 leader_election.cc:304] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:38.967482 30165 raft_consensus.cc:2804] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:38.967563 30165 raft_consensus.cc:493] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:38.967618 30165 raft_consensus.cc:3060] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.968230 28549 raft_consensus.cc:3060] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.970216 30165 raft_consensus.cc:515] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.970692 29746 ts_tablet_manager.cc:1929] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:38.971204 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dd0eaa64271489d92aa380df877f257" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:38.971427 28680 raft_consensus.cc:3060] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:38.971760 28799 tablet_bootstrap.cc:492] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:38.973483 28799 tablet_bootstrap.cc:654] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.974102 28680 raft_consensus.cc:2468] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:38.973920 29745 ts_tablet_manager.cc:1929] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:38.983139 29745 log.cc:1199] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/35d8dd2c388e43a0be380aa953ec2c82
I20260618 13:59:38.983541 29745 ts_tablet_manager.cc:1950] T 35d8dd2c388e43a0be380aa953ec2c82 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:38.983722 29566 tablet_replica.cc:333] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:38.983979 29566 raft_consensus.cc:2243] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:38.984248 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 131) took 4354 ms. Trace:
I20260618 13:59:38.984536 29566 raft_consensus.cc:2272] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:38.985483 28799 tablet_bootstrap.cc:492] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:38.985595 28799 ts_tablet_manager.cc:1403] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20260618 13:59:38.992323 28797 tablet_bootstrap.cc:492] T 340f6bbf18eb412ca3afec237ff7752d P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:38.993269 28799 raft_consensus.cc:359] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.993468 28799 raft_consensus.cc:385] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.993541 28799 raft_consensus.cc:740] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.993749 28799 consensus_queue.cc:260] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.993844 28797 ts_tablet_manager.cc:1403] T 340f6bbf18eb412ca3afec237ff7752d P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.065s user 0.002s sys 0.001s
I20260618 13:59:38.994277 28799 ts_tablet_manager.cc:1434] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260618 13:59:38.994544 28799 tablet_bootstrap.cc:492] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:38.995182 28797 raft_consensus.cc:359] T 340f6bbf18eb412ca3afec237ff7752d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:38.996348 28799 tablet_bootstrap.cc:654] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:38.971040 30165 leader_election.cc:290] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:38.971783 28483 leader_election.cc:343] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 3dd0eaa64271489d92aa380df877f257
I20260618 13:59:38.997345 29746 log.cc:1199] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/dca54111fea047d9ae0fabe376d89382
I20260618 13:59:38.997838 29746 ts_tablet_manager.cc:1950] T dca54111fea047d9ae0fabe376d89382 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:38.998991 28484 leader_election.cc:304] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:38.996975 29566 ts_tablet_manager.cc:1916] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:38.999321 28797 raft_consensus.cc:385] T 340f6bbf18eb412ca3afec237ff7752d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:38.999493 30165 raft_consensus.cc:2804] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:38.999531 28797 raft_consensus.cc:740] T 340f6bbf18eb412ca3afec237ff7752d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:38.999631 30165 raft_consensus.cc:697] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:38.999825 30165 consensus_queue.cc:237] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.001197 28797 consensus_queue.cc:260] T 340f6bbf18eb412ca3afec237ff7752d P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.002552 30391 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a950ab7cd66b4cb0b29592aa692c2dd0)
W20260618 13:59:39.004354 28617 consensus_peers.cc:597] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 75824a5d2f7e47838b60e6a2b785c6b2. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.004743 30391 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:39.005228 28797 ts_tablet_manager.cc:1434] T 340f6bbf18eb412ca3afec237ff7752d P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260618 13:59:39.005524 28797 tablet_bootstrap.cc:492] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:39.006858 29048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7637285570574a6dbcef1de6f84ecac0" candidate_term: 3 candidate_status { last_received { term: 2 index: 624 } } ignore_live_leader: false dest_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" is_pre_election: true
I20260618 13:59:39.006340 30391 leader_election.cc:290] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e4d4f4566be42dfbafb7c0e40fb17ff (127.27.100.62:43001), a950ab7cd66b4cb0b29592aa692c2dd0 (127.27.100.61:45967)
I20260618 13:59:39.011914 28097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7637285570574a6dbcef1de6f84ecac0" candidate_term: 3 candidate_status { last_received { term: 2 index: 624 } } ignore_live_leader: false dest_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" is_pre_election: true
I20260618 13:59:39.012573 28799 tablet_bootstrap.cc:492] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:39.012709 28799 ts_tablet_manager.cc:1403] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.001s
I20260618 13:59:39.012687 29800 leader_election.cc:304] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7637285570574a6dbcef1de6f84ecac0; no voters: 9e4d4f4566be42dfbafb7c0e40fb17ff, a950ab7cd66b4cb0b29592aa692c2dd0
I20260618 13:59:39.013499 30391 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 7637285570574a6dbcef1de6f84ecac0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260618 13:59:39.013512 28799 raft_consensus.cc:359] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.013677 28799 raft_consensus.cc:385] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.013732 28799 raft_consensus.cc:740] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.013976 28799 consensus_queue.cc:260] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.014918 28799 ts_tablet_manager.cc:1434] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:39.015194 28799 tablet_bootstrap.cc:492] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:39.016227 28798 tablet_bootstrap.cc:492] T f54910015987484e95b295352d859315 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:39.016427 28798 ts_tablet_manager.cc:1403] T f54910015987484e95b295352d859315 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.056s user 0.001s sys 0.003s
I20260618 13:59:39.016987 28799 tablet_bootstrap.cc:654] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.019189 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 120) took 4372 ms. Trace:
I20260618 13:59:39.019356 29746 rpcz_store.cc:276] 0618 13:59:34.647143 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.647189 (+ 46us) service_pool.cc:224] Handling call
0618 13:59:38.839263 (+4192074us) ts_tablet_manager.cc:1151] Deleting tablet dca54111fea047d9ae0fabe376d89382
0618 13:59:38.839276 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.921270 (+ 81994us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.970632 (+ 49362us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.019160 (+ 48528us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4191092}
I20260618 13:59:39.020008 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet dca54111fea047d9ae0fabe376d89382 (table default.Table_77ced729210e46da993ea9e57db1d716 [id=fcccef3123f14001be3f111806a209a0]) successfully deleted
I20260618 13:59:39.020669 29746 tablet_replica.cc:333] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:39.020926 29746 raft_consensus.cc:2243] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:39.021164 29746 raft_consensus.cc:2272] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:39.022064 29746 ts_tablet_manager.cc:1916] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:39.037077 28617 consensus_peers.cc:597] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e6de1293572f4e75af3417f281b9a142. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.040359 28799 tablet_bootstrap.cc:492] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:39.040736 28799 ts_tablet_manager.cc:1403] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.000s
I20260618 13:59:39.041568 28799 raft_consensus.cc:359] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.041755 28799 raft_consensus.cc:385] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.041812 28799 raft_consensus.cc:740] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.042147 28799 consensus_queue.cc:260] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.042531 29745 rpcz_store.cc:276] 0618 13:59:34.629886 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.629943 (+ 57us) service_pool.cc:224] Handling call
0618 13:59:38.953391 (+4323448us) ts_tablet_manager.cc:1151] Deleting tablet 35d8dd2c388e43a0be380aa953ec2c82
0618 13:59:38.953399 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:38.957675 (+ 4276us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.973857 (+ 16182us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:38.984231 (+ 10374us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4323320}
I20260618 13:59:39.043073 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 35d8dd2c388e43a0be380aa953ec2c82 (table default.Table_a8705c5cd49a40049ff1020a01aa7f5b [id=f9ff809bb5da4d048e7520cec3316f6d]) successfully deleted
I20260618 13:59:39.044251 28799 ts_tablet_manager.cc:1434] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260618 13:59:39.050908 28799 tablet_bootstrap.cc:492] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:39.051146 30151 raft_consensus.cc:493] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.054540 30151 raft_consensus.cc:515] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.055130 30151 leader_election.cc:290] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:39.055583 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6216474b74a14af6abf977e312b77" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:39.056500 28346 leader_election.cc:343] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.056648 28799 tablet_bootstrap.cc:654] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.057338 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6216474b74a14af6abf977e312b77" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:39.058149 28346 leader_election.cc:343] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.058279 28346 leader_election.cc:304] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.058665 30151 raft_consensus.cc:2749] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.060688 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 132) took 4430 ms. Trace:
I20260618 13:59:39.060791 29745 rpcz_store.cc:276] 0618 13:59:34.630174 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.630217 (+ 43us) service_pool.cc:224] Handling call
0618 13:59:39.043538 (+4413321us) ts_tablet_manager.cc:1151] Deleting tablet 316f470ea82f42fca48e4e96d49c9574
0618 13:59:39.044071 (+ 533us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.060668 (+ 16597us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4413188}
I20260618 13:59:39.061151 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 133) took 4430 ms. Trace:
W20260618 13:59:39.061280 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 316f470ea82f42fca48e4e96d49c9574 because the tablet was not found. No further retry: Not found: Tablet not found: 316f470ea82f42fca48e4e96d49c9574
I20260618 13:59:39.061225 29745 rpcz_store.cc:276] 0618 13:59:34.630585 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.630635 (+ 50us) service_pool.cc:224] Handling call
0618 13:59:39.061074 (+4430439us) ts_tablet_manager.cc:1151] Deleting tablet d3dbf13544ce408ab97834aec5862390
0618 13:59:39.061085 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.061137 (+ 52us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4430277}
I20260618 13:59:39.061530 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 135) took 4430 ms. Trace:
W20260618 13:59:39.061617 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet d3dbf13544ce408ab97834aec5862390 because the tablet was not found. No further retry: Not found: Tablet not found: d3dbf13544ce408ab97834aec5862390
I20260618 13:59:39.061604 29745 rpcz_store.cc:276] 0618 13:59:34.631258 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.631311 (+ 53us) service_pool.cc:224] Handling call
0618 13:59:39.061454 (+4430143us) ts_tablet_manager.cc:1151] Deleting tablet fddea8fba4da40d7a5780e6750caf5e9
0618 13:59:39.061463 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.061517 (+ 54us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4425691}
I20260618 13:59:39.061873 29745 tablet_replica.cc:333] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
W20260618 13:59:39.062386 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet fddea8fba4da40d7a5780e6750caf5e9 because the tablet was not found. No further retry: Not found: Tablet not found: fddea8fba4da40d7a5780e6750caf5e9
I20260618 13:59:39.063756 29745 raft_consensus.cc:2243] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:39.063928 29745 raft_consensus.cc:2272] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:39.064419 29745 ts_tablet_manager.cc:1916] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:39.071252 28618 consensus_peers.cc:597] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.071728 29745 ts_tablet_manager.cc:1929] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:39.071843 29745 log.cc:1199] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/9c17e76ae9b44207998a43e3576908c4
I20260618 13:59:39.072750 29745 ts_tablet_manager.cc:1950] T 9c17e76ae9b44207998a43e3576908c4 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:39.073661 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 136) took 4288 ms. Trace:
I20260618 13:59:39.073751 29745 rpcz_store.cc:276] 0618 13:59:34.785390 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.785459 (+ 69us) service_pool.cc:224] Handling call
0618 13:59:39.061830 (+4276371us) ts_tablet_manager.cc:1151] Deleting tablet 9c17e76ae9b44207998a43e3576908c4
0618 13:59:39.061839 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.068469 (+ 6630us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.071673 (+ 3204us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.073640 (+ 1967us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1687,"tablet-delete.queue_time_us":4276176}
I20260618 13:59:39.074149 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 9c17e76ae9b44207998a43e3576908c4 (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]) successfully deleted
I20260618 13:59:39.074717 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 139) took 4066 ms. Trace:
I20260618 13:59:39.074800 29745 rpcz_store.cc:276] 0618 13:59:35.008746 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.008804 (+ 58us) service_pool.cc:224] Handling call
0618 13:59:39.074606 (+4065802us) ts_tablet_manager.cc:1151] Deleting tablet dd2385e6752d49429422fc854539e9c6
0618 13:59:39.074620 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.074696 (+ 76us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4065583}
I20260618 13:59:39.075229 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 141) took 4023 ms. Trace:
W20260618 13:59:39.075213 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet dd2385e6752d49429422fc854539e9c6 because the tablet was not found. No further retry: Not found: Tablet not found: dd2385e6752d49429422fc854539e9c6
I20260618 13:59:39.075294 29745 rpcz_store.cc:276] 0618 13:59:35.051467 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.051785 (+ 318us) service_pool.cc:224] Handling call
0618 13:59:39.075151 (+4023366us) ts_tablet_manager.cc:1151] Deleting tablet 283523e5b22843ba935eabe6283d3a99
0618 13:59:39.075161 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.075215 (+ 54us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4023158}
I20260618 13:59:39.075634 29745 tablet_replica.cc:333] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
W20260618 13:59:39.075845 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 283523e5b22843ba935eabe6283d3a99 because the tablet was not found. No further retry: Not found: Tablet not found: 283523e5b22843ba935eabe6283d3a99
I20260618 13:59:39.075948 29745 raft_consensus.cc:2243] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:39.076113 29745 raft_consensus.cc:2272] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:39.076581 29745 ts_tablet_manager.cc:1916] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:39.079762 30274 raft_consensus.cc:493] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.079883 30274 raft_consensus.cc:515] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.080447 30274 leader_election.cc:290] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.081916 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d1073b726ef4cbeac6c874fc8863e90" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:39.082062 28403 raft_consensus.cc:2468] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:39.083092 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d1073b726ef4cbeac6c874fc8863e90" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:39.083761 28618 leader_election.cc:343] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.083951 29745 ts_tablet_manager.cc:1929] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:39.084062 29745 log.cc:1199] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/f45556a59c424ca2bc30956566374c89
I20260618 13:59:39.084400 29745 ts_tablet_manager.cc:1950] T f45556a59c424ca2bc30956566374c89 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:39.085017 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 145) took 3670 ms. Trace:
I20260618 13:59:39.085170 28617 leader_election.cc:304] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:39.085107 29745 rpcz_store.cc:276] 0618 13:59:35.414629 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.414689 (+ 60us) service_pool.cc:224] Handling call
0618 13:59:39.075594 (+3660905us) ts_tablet_manager.cc:1151] Deleting tablet f45556a59c424ca2bc30956566374c89
0618 13:59:39.075602 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.079467 (+ 3865us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.083897 (+ 4430us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.085001 (+ 1104us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3626160}
I20260618 13:59:39.085595 29745 tablet_replica.cc:333] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:39.085559 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet f45556a59c424ca2bc30956566374c89 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]) successfully deleted
I20260618 13:59:39.085644 29644 raft_consensus.cc:2804] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:39.085714 29644 raft_consensus.cc:493] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:39.085786 29644 raft_consensus.cc:3060] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.085781 29745 raft_consensus.cc:2243] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:39.085937 29745 raft_consensus.cc:2272] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:39.086503 29745 ts_tablet_manager.cc:1916] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:39.093626 29745 ts_tablet_manager.cc:1929] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:39.093770 29745 log.cc:1199] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/98446f40214c4ac984fcfe98310005c5
I20260618 13:59:39.093932 29644 raft_consensus.cc:515] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.094107 29745 ts_tablet_manager.cc:1950] T 98446f40214c4ac984fcfe98310005c5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:39.095255 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d1073b726ef4cbeac6c874fc8863e90" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:39.095688 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 147) took 3395 ms. Trace:
W20260618 13:59:39.095777 28618 leader_election.cc:343] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.095816 29745 rpcz_store.cc:276] 0618 13:59:35.700015 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.700079 (+ 64us) service_pool.cc:224] Handling call
0618 13:59:39.085551 (+3385472us) ts_tablet_manager.cc:1151] Deleting tablet 98446f40214c4ac984fcfe98310005c5
0618 13:59:39.085561 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.089905 (+ 4344us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.093564 (+ 3659us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.095665 (+ 2101us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3385273}
I20260618 13:59:39.096479 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 98446f40214c4ac984fcfe98310005c5 (table default.Table_83d021ed148243ffb453cb5e2484b7b4 [id=b3cae943ecdf410ca1cf711c5ddfe14a]) successfully deleted
I20260618 13:59:39.096539 29745 tablet_replica.cc:333] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:39.096943 29745 raft_consensus.cc:2243] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:39.097191 29644 leader_election.cc:290] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.097314 29745 raft_consensus.cc:2272] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:39.097801 29745 ts_tablet_manager.cc:1916] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:39.097886 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d1073b726ef4cbeac6c874fc8863e90" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:39.098029 28403 raft_consensus.cc:3060] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.102967 29745 ts_tablet_manager.cc:1929] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:39.103473 29745 log.cc:1199] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/cea4bc13eabc4dd98b4f96723498014c
I20260618 13:59:39.103853 29745 ts_tablet_manager.cc:1950] T cea4bc13eabc4dd98b4f96723498014c P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:39.104648 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 159) took 1692 ms. Trace:
I20260618 13:59:39.104733 29745 rpcz_store.cc:276] 0618 13:59:37.411964 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:37.412025 (+ 61us) service_pool.cc:224] Handling call
0618 13:59:39.096491 (+1684466us) ts_tablet_manager.cc:1151] Deleting tablet cea4bc13eabc4dd98b4f96723498014c
0618 13:59:39.096502 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.100367 (+ 3865us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.102914 (+ 2547us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.104629 (+ 1715us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1684225}
I20260618 13:59:39.105391 29745 tablet_replica.cc:333] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:39.105357 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet cea4bc13eabc4dd98b4f96723498014c (table default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c]) successfully deleted
I20260618 13:59:39.105576 29745 raft_consensus.cc:2243] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:39.105753 29745 raft_consensus.cc:2272] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:39.101630 28403 raft_consensus.cc:2468] T 2d1073b726ef4cbeac6c874fc8863e90 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:39.106745 29745 ts_tablet_manager.cc:1916] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:39.107101 28617 leader_election.cc:304] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:39.107559 29644 raft_consensus.cc:2804] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:39.107733 29644 raft_consensus.cc:697] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:39.107981 29644 consensus_queue.cc:237] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.108628 30151 raft_consensus.cc:493] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.108736 30151 raft_consensus.cc:515] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.109278 30151 leader_election.cc:290] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:39.110176 30152 raft_consensus.cc:493] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.110307 30152 raft_consensus.cc:515] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.110283 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c4a31a4f202473d97b17b5b5dfdb615" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:39.110984 28346 leader_election.cc:343] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.111176 30151 raft_consensus.cc:493] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.111291 30151 raft_consensus.cc:515] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.111776 30152 leader_election.cc:290] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:39.111907 30151 leader_election.cc:290] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:39.112629 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c4a31a4f202473d97b17b5b5dfdb615" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:39.112629 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2d54a901e9f4d5c89700979ce0618bf" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:39.112713 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1968e5ce2e42a5b836c90b0904f158" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:39.112830 28681 raft_consensus.cc:2468] T d2d54a901e9f4d5c89700979ce0618bf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:39.113094 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1968e5ce2e42a5b836c90b0904f158" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:39.113535 28346 leader_election.cc:343] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.113749 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2d54a901e9f4d5c89700979ce0618bf" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:39.114153 28346 leader_election.cc:343] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:39.116243 28346 leader_election.cc:343] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.116405 28346 leader_election.cc:304] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.116801 28346 leader_election.cc:304] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:39.117058 30152 raft_consensus.cc:2749] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:39.117147 28346 leader_election.cc:343] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.117247 30152 raft_consensus.cc:2804] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:39.117264 28346 leader_election.cc:304] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.117308 30152 raft_consensus.cc:493] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:39.117450 30152 raft_consensus.cc:3060] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.119979 30152 raft_consensus.cc:515] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.120570 30152 leader_election.cc:290] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:39.120735 30152 raft_consensus.cc:2749] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.121507 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2d54a901e9f4d5c89700979ce0618bf" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:39.121959 28346 leader_election.cc:343] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:39.124059 28618 consensus_peers.cc:597] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.125110 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2d54a901e9f4d5c89700979ce0618bf" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:39.125330 28682 raft_consensus.cc:3060] T d2d54a901e9f4d5c89700979ce0618bf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.126197 28798 raft_consensus.cc:359] T f54910015987484e95b295352d859315 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.126376 28798 raft_consensus.cc:385] T f54910015987484e95b295352d859315 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.126691 28798 raft_consensus.cc:740] T f54910015987484e95b295352d859315 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.128134 28682 raft_consensus.cc:2468] T d2d54a901e9f4d5c89700979ce0618bf P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:39.135144 28346 leader_election.cc:304] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:39.135250 28797 tablet_bootstrap.cc:654] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.135762 30151 raft_consensus.cc:2804] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:39.135900 30151 raft_consensus.cc:697] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:39.136194 30151 consensus_queue.cc:237] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.136639 28798 consensus_queue.cc:260] T f54910015987484e95b295352d859315 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.137339 28798 ts_tablet_manager.cc:1434] T f54910015987484e95b295352d859315 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.121s user 0.000s sys 0.001s
W20260618 13:59:39.143177 28617 consensus_peers.cc:597] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 827cc0987bd541759e334fb620f04ed5. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.154955 28798 tablet_bootstrap.cc:492] T 15483938a06e459789f87c6bb456bd15 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:39.156555 28798 tablet_bootstrap.cc:654] T 15483938a06e459789f87c6bb456bd15 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.164381 29745 ts_tablet_manager.cc:1929] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:39.164494 29745 log.cc:1199] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/c01c8463055e4526b3c75190066f30b6
I20260618 13:59:39.164745 29745 ts_tablet_manager.cc:1950] T c01c8463055e4526b3c75190066f30b6 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:39.165004 29566 ts_tablet_manager.cc:1929] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:39.165187 29566 log.cc:1199] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/aa8cf927bed9437086efd8e0f21ba26a
I20260618 13:59:39.165421 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 160) took 1666 ms. Trace:
I20260618 13:59:39.165511 29745 rpcz_store.cc:276] 0618 13:59:37.499372 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:37.499420 (+ 48us) service_pool.cc:224] Handling call
0618 13:59:39.105332 (+1605912us) ts_tablet_manager.cc:1151] Deleting tablet c01c8463055e4526b3c75190066f30b6
0618 13:59:39.105342 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.157146 (+ 51804us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.164313 (+ 7167us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.165402 (+ 1089us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1605736}
I20260618 13:59:39.165652 29566 ts_tablet_manager.cc:1950] T aa8cf927bed9437086efd8e0f21ba26a P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:39.165799 29745 tablet_replica.cc:333] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:39.165936 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet c01c8463055e4526b3c75190066f30b6 (table default.Table_7e5d58ad3e8043919b66e8890819535c [id=96aff34b68c1494f9228a11eb38de4c7]) successfully deleted
I20260618 13:59:39.165997 29745 raft_consensus.cc:2243] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:39.166126 29745 raft_consensus.cc:2272] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:39.167670 28799 tablet_bootstrap.cc:492] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:39.167804 28799 ts_tablet_manager.cc:1403] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.117s user 0.011s sys 0.024s
I20260618 13:59:39.168493 28799 raft_consensus.cc:359] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.168656 28799 raft_consensus.cc:385] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.168716 28799 raft_consensus.cc:740] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.168967 28799 consensus_queue.cc:260] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.169497 28799 ts_tablet_manager.cc:1434] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:39.169761 28799 tablet_bootstrap.cc:492] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:39.169858 29745 ts_tablet_manager.cc:1916] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:39.171137 28799 tablet_bootstrap.cc:654] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.172423 28797 tablet_bootstrap.cc:492] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:39.172556 28797 ts_tablet_manager.cc:1403] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.167s user 0.002s sys 0.002s
I20260618 13:59:39.177769 28797 raft_consensus.cc:359] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.178356 28797 raft_consensus.cc:385] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.178463 28797 raft_consensus.cc:740] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.178691 28797 consensus_queue.cc:260] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.179252 28797 ts_tablet_manager.cc:1434] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260618 13:59:39.179889 29745 ts_tablet_manager.cc:1929] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:39.180006 29745 log.cc:1199] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/6c4a31a4f202473d97b17b5b5dfdb615
W20260618 13:59:39.180114 28617 consensus_peers.cc:597] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 02b5cc08ca0c4e719e0414b345211ba5. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.180299 29745 ts_tablet_manager.cc:1950] T 6c4a31a4f202473d97b17b5b5dfdb615 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:39.180437 28797 tablet_bootstrap.cc:492] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:39.181064 29745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38230 (request call id 161) took 1661 ms. Trace:
I20260618 13:59:39.181162 29745 rpcz_store.cc:276] 0618 13:59:37.519604 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:37.519657 (+ 53us) service_pool.cc:224] Handling call
0618 13:59:39.165758 (+1646101us) ts_tablet_manager.cc:1151] Deleting tablet 6c4a31a4f202473d97b17b5b5dfdb615
0618 13:59:39.165766 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.172910 (+ 7144us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.179832 (+ 6922us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.181043 (+ 1211us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1645951}
I20260618 13:59:39.181854 30151 raft_consensus.cc:493] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.181999 30151 raft_consensus.cc:515] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.182629 30151 leader_election.cc:290] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:39.182687 28797 tablet_bootstrap.cc:654] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.183084 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 6c4a31a4f202473d97b17b5b5dfdb615 (table default.Table_152a483501b54a67bbdc0cae4959d452 [id=daf2b982b88e4021b327a0e52cf5316c]) successfully deleted
I20260618 13:59:39.183560 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e9a417eecf4d33ac3bd9bd6325f447" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:39.184218 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e9a417eecf4d33ac3bd9bd6325f447" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:39.184333 28798 tablet_bootstrap.cc:492] T 15483938a06e459789f87c6bb456bd15 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:39.184466 28798 ts_tablet_manager.cc:1403] T 15483938a06e459789f87c6bb456bd15 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.030s user 0.000s sys 0.003s
W20260618 13:59:39.184763 28346 leader_election.cc:343] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:39.184966 28346 leader_election.cc:343] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.185055 28346 leader_election.cc:304] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.185178 28798 raft_consensus.cc:359] T 15483938a06e459789f87c6bb456bd15 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:39.185338 28798 raft_consensus.cc:385] T 15483938a06e459789f87c6bb456bd15 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.185417 28798 raft_consensus.cc:740] T 15483938a06e459789f87c6bb456bd15 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.185568 30152 raft_consensus.cc:2749] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.185609 28798 consensus_queue.cc:260] T 15483938a06e459789f87c6bb456bd15 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:39.186107 28798 ts_tablet_manager.cc:1434] T 15483938a06e459789f87c6bb456bd15 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:39.186374 28798 tablet_bootstrap.cc:492] T 957670718c8c4b8cb87115cc4a2fa861 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:39.187971 28798 tablet_bootstrap.cc:654] T 957670718c8c4b8cb87115cc4a2fa861 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.190863 30165 raft_consensus.cc:493] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:39.191000 30165 raft_consensus.cc:515] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:39.191687 30165 leader_election.cc:290] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:39.192270 28484 leader_election.cc:343] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: bce7bf74b6d24d0bb43808b8a62cf1b7
W20260618 13:59:39.192726 28483 leader_election.cc:343] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: bce7bf74b6d24d0bb43808b8a62cf1b7
I20260618 13:59:39.192849 28483 leader_election.cc:304] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.193272 30165 raft_consensus.cc:2749] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:39.199070 28540 raft_consensus.cc:3060] T 41c92a594cd64982a8972e4a019d1cd8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.202220 29303 consensus_queue.cc:1048] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:39.205364 30165 raft_consensus.cc:493] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.205514 30165 raft_consensus.cc:515] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:39.206207 30165 leader_election.cc:290] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:39.207257 28483 leader_election.cc:343] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 68b8293a7c924a449307ab88a1cf9290
I20260618 13:59:39.211364 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 92) took 7096 ms. Trace:
I20260618 13:59:39.211465 29566 rpcz_store.cc:276] 0618 13:59:32.114927 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.114968 (+ 41us) service_pool.cc:224] Handling call
0618 13:59:38.983664 (+6868696us) ts_tablet_manager.cc:1151] Deleting tablet aa8cf927bed9437086efd8e0f21ba26a
0618 13:59:38.983676 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.128119 (+144443us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.164936 (+ 36817us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.211340 (+ 46404us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6868562}
I20260618 13:59:39.211983 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet aa8cf927bed9437086efd8e0f21ba26a (table default.Table_cf80fd5bbdde4cfb855c8fbe860a15f3 [id=9d63ad7327a44124af817b465a9ae0da]) successfully deleted
I20260618 13:59:39.212756 29566 tablet_replica.cc:333] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:39.212978 29566 raft_consensus.cc:2243] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:39.213280 29566 raft_consensus.cc:2272] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:39.213690 28484 leader_election.cc:343] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 68b8293a7c924a449307ab88a1cf9290
I20260618 13:59:39.213819 28484 leader_election.cc:304] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:39.213848 28617 consensus_peers.cc:597] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cd06bb7cffc34412b420d70435fce373. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.214407 30165 raft_consensus.cc:2749] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.214984 29746 ts_tablet_manager.cc:1929] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260618 13:59:39.215095 29746 log.cc:1199] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/aef7c1f111634c98af046e73fa9ffdeb
I20260618 13:59:39.215456 29746 ts_tablet_manager.cc:1950] T aef7c1f111634c98af046e73fa9ffdeb P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:39.216118 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 121) took 4562 ms. Trace:
I20260618 13:59:39.216208 29746 rpcz_store.cc:276] 0618 13:59:34.653327 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.653374 (+ 47us) service_pool.cc:224] Handling call
0618 13:59:39.020602 (+4367228us) ts_tablet_manager.cc:1151] Deleting tablet aef7c1f111634c98af046e73fa9ffdeb
0618 13:59:39.020615 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.148253 (+127638us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.214928 (+ 66675us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.216100 (+ 1172us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4367046}
I20260618 13:59:39.216670 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet aef7c1f111634c98af046e73fa9ffdeb (table default.Table_191169025bff42cab93cf42503fa7220 [id=9c29cca38acc493aaec62cf3b0d93fcf]) successfully deleted
I20260618 13:59:39.217537 29566 ts_tablet_manager.cc:1916] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:39.222335 28617 consensus_peers.cc:597] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab521028ac1a4d0da013112d76bb169f. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.218940 28797 tablet_bootstrap.cc:492] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:39.223980 28797 ts_tablet_manager.cc:1403] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.044s user 0.003s sys 0.000s
I20260618 13:59:39.224946 28797 raft_consensus.cc:359] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.229290 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:39.230024 28797 raft_consensus.cc:385] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.230206 28797 raft_consensus.cc:740] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.230660 29746 tablet_replica.cc:333] T 75824a5d2f7e47838b60e6a2b785c6b2 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:39.230916 28797 consensus_queue.cc:260] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.231935 29746 raft_consensus.cc:2243] T 75824a5d2f7e47838b60e6a2b785c6b2 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:39.232786 28797 ts_tablet_manager.cc:1434] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20260618 13:59:39.233162 29746 raft_consensus.cc:2272] T 75824a5d2f7e47838b60e6a2b785c6b2 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:39.236490 29746 ts_tablet_manager.cc:1916] T 75824a5d2f7e47838b60e6a2b785c6b2 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:39.237049 28797 tablet_bootstrap.cc:492] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:39.245200 28617 consensus_peers.cc:597] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7572d17724d249739be1a1cb9468f909. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.248673 29644 raft_consensus.cc:493] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.248818 29644 raft_consensus.cc:515] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.249511 29644 leader_election.cc:290] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.251263 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f7eaa7d218440ca2c35340c5c87814" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:39.251480 28617 leader_election.cc:343] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: e5f7eaa7d218440ca2c35340c5c87814
W20260618 13:59:39.251830 28618 leader_election.cc:343] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.251935 28618 leader_election.cc:304] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:39.252139 29644 raft_consensus.cc:2749] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.258689 28797 tablet_bootstrap.cc:654] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.264511 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:39.265390 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 2128dbec33db453ba51070767a15b21e (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]). Expected version 2 got 0
I20260618 13:59:39.266150 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:39.268775 28799 tablet_bootstrap.cc:492] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:39.268936 28799 ts_tablet_manager.cc:1403] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.099s user 0.004s sys 0.005s
I20260618 13:59:39.269737 28799 raft_consensus.cc:359] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.269908 28799 raft_consensus.cc:385] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.269970 28799 raft_consensus.cc:740] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.270210 28799 consensus_queue.cc:260] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.276639 28048 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu pid 29476
I20260618 13:59:39.279134 28799 ts_tablet_manager.cc:1434] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260618 13:59:39.279680 28799 tablet_bootstrap.cc:492] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:39.281586 28799 tablet_bootstrap.cc:654] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.282877 30027 consensus_queue.cc:1048] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:39.310256 30027 consensus_queue.cc:1048] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:39.314479 30165 raft_consensus.cc:493] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:39.314643 30165 raft_consensus.cc:515] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.315327 30165 leader_election.cc:290] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:39.317233 28483 leader_election.cc:343] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: e95c88e4b0a046a3ad69ee49af47c7c3
I20260618 13:59:39.275288 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:39.318943 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:39.319150 28540 raft_consensus.cc:3060] T 9ab1ddf79b094ba2aeee9225c0b332c3 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.320994 29746 ts_tablet_manager.cc:1929] T 75824a5d2f7e47838b60e6a2b785c6b2 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:39.321090 29746 log.cc:1199] T 75824a5d2f7e47838b60e6a2b785c6b2 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/75824a5d2f7e47838b60e6a2b785c6b2
I20260618 13:59:39.321427 29746 ts_tablet_manager.cc:1950] T 75824a5d2f7e47838b60e6a2b785c6b2 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:39.322119 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 122) took 4668 ms. Trace:
I20260618 13:59:39.322199 29746 rpcz_store.cc:276] 0618 13:59:34.653768 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.653829 (+ 61us) service_pool.cc:224] Handling call
0618 13:59:39.230604 (+4576775us) ts_tablet_manager.cc:1151] Deleting tablet 75824a5d2f7e47838b60e6a2b785c6b2
0618 13:59:39.230615 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.269758 (+ 39143us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.320935 (+ 51177us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.322101 (+ 1166us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4576596}
I20260618 13:59:39.322623 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 75824a5d2f7e47838b60e6a2b785c6b2 (table default.Table_d838306823ff4e22a68efe0a8cb751ff [id=bbfd0a6a47db4b1b8c534405b1068d5e]) successfully deleted
I20260618 13:59:39.319221 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
W20260618 13:59:39.323017 28484 leader_election.cc:343] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: e95c88e4b0a046a3ad69ee49af47c7c3
I20260618 13:59:39.323133 28484 leader_election.cc:304] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.324030 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:39.324575 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:39.324993 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:39.325138 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:39.325289 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:39.326063 28797 tablet_bootstrap.cc:492] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:39.326187 28797 ts_tablet_manager.cc:1403] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.089s user 0.003s sys 0.002s
I20260618 13:59:39.326890 28797 raft_consensus.cc:359] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.327056 28797 raft_consensus.cc:385] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.327103 28797 raft_consensus.cc:740] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.327304 28797 consensus_queue.cc:260] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.327404 30338 raft_consensus.cc:2749] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:39.327790 28797 ts_tablet_manager.cc:1434] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:39.328061 28797 tablet_bootstrap.cc:492] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:39.329849 28797 tablet_bootstrap.cc:654] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.331089 28798 tablet_bootstrap.cc:492] T 957670718c8c4b8cb87115cc4a2fa861 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:39.331200 28798 ts_tablet_manager.cc:1403] T 957670718c8c4b8cb87115cc4a2fa861 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.145s user 0.002s sys 0.006s
I20260618 13:59:39.331864 28798 raft_consensus.cc:359] T 957670718c8c4b8cb87115cc4a2fa861 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.332017 28798 raft_consensus.cc:385] T 957670718c8c4b8cb87115cc4a2fa861 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.332072 28798 raft_consensus.cc:740] T 957670718c8c4b8cb87115cc4a2fa861 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.332288 28798 consensus_queue.cc:260] T 957670718c8c4b8cb87115cc4a2fa861 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.332751 28798 ts_tablet_manager.cc:1434] T 957670718c8c4b8cb87115cc4a2fa861 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260618 13:59:39.332998 28798 tablet_bootstrap.cc:492] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:39.334699 28798 tablet_bootstrap.cc:654] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.335434 28799 tablet_bootstrap.cc:492] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:39.335578 28799 ts_tablet_manager.cc:1403] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.056s user 0.001s sys 0.002s
I20260618 13:59:39.339625 28799 raft_consensus.cc:359] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.339792 28799 raft_consensus.cc:385] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.339907 29746 tablet_replica.cc:333] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:39.340065 29746 raft_consensus.cc:2243] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:39.340273 29746 raft_consensus.cc:2272] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:39.340823 29746 ts_tablet_manager.cc:1916] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:39.341578 28617 consensus_peers.cc:597] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c3d523419ff244f085d6eac1a3231973. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.342789 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 72c04f55890648f0a8184f3783243f7c (table default.Table_773be314d7e741fcb4fb6eb7d954802e [id=0d83fc68255c458587390c58b12e00d7]). Expected version 2 got 0
I20260618 13:59:39.343377 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:39.343776 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a]). Expected version 1 got 0
I20260618 13:59:39.344146 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:39.344470 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:39.344802 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:39.345055 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:39.345324 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:39.339856 28799 raft_consensus.cc:740] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.345784 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:39.345990 28799 consensus_queue.cc:260] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.346143 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:39.346563 28799 ts_tablet_manager.cc:1434] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260618 13:59:39.346704 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:39.346906 28799 tablet_bootstrap.cc:492] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:39.348170 28799 tablet_bootstrap.cc:654] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.355268 29930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 13:59:39.365216 29566 ts_tablet_manager.cc:1929] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:39.365384 29566 log.cc:1199] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/bc0adedecee4405e86b0a37b9e62e352
I20260618 13:59:39.365805 29566 ts_tablet_manager.cc:1950] T bc0adedecee4405e86b0a37b9e62e352 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
W20260618 13:59:39.368973 28617 consensus_peers.cc:597] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 98b42d23832545ae81a7cdff19e4689f. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.370736 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 94) took 7241 ms. Trace:
I20260618 13:59:39.371503 29566 rpcz_store.cc:276] 0618 13:59:32.129431 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.129476 (+ 45us) service_pool.cc:224] Handling call
0618 13:59:39.212704 (+7083228us) ts_tablet_manager.cc:1151] Deleting tablet bc0adedecee4405e86b0a37b9e62e352
0618 13:59:39.212714 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.234400 (+ 21686us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.365156 (+130756us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.370719 (+ 5563us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7083035}
I20260618 13:59:39.372097 29566 tablet_replica.cc:333] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:39.372603 28798 tablet_bootstrap.cc:492] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:39.372742 28798 ts_tablet_manager.cc:1403] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.000s
I20260618 13:59:39.372954 29566 raft_consensus.cc:2243] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:39.373466 28798 raft_consensus.cc:359] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.373638 28798 raft_consensus.cc:385] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.373700 28798 raft_consensus.cc:740] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.374768 28797 tablet_bootstrap.cc:492] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:39.374888 28797 ts_tablet_manager.cc:1403] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.047s user 0.003s sys 0.000s
I20260618 13:59:39.375093 29566 raft_consensus.cc:2272] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:39.376008 28797 raft_consensus.cc:359] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:39.376161 28797 raft_consensus.cc:385] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.376219 28797 raft_consensus.cc:740] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.377517 29566 ts_tablet_manager.cc:1916] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:39.380587 28798 consensus_queue.cc:260] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.381132 28798 ts_tablet_manager.cc:1434] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260618 13:59:39.372059 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet bc0adedecee4405e86b0a37b9e62e352 (table default.Table_cc20b3f971bb46969baf1fc71e217f2c [id=0a955d9a7fec4cd7985ac56e210fe8a6]) successfully deleted
I20260618 13:59:39.384757 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:39.384999 30338 raft_consensus.cc:493] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:39.385143 30338 raft_consensus.cc:515] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:39.385800 30338 leader_election.cc:290] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.386127 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:39.386385 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:39.386690 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:39.386883 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:39.387209 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_c70d6c5b0dc94b33b4c5772edd2b9dca [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 2 got 0
W20260618 13:59:39.387311 28484 leader_election.cc:343] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: b0285c75c632457383e81805ab036aaf
I20260618 13:59:39.387518 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:39.387782 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:39.388165 29018 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
W20260618 13:59:39.388525 28483 leader_election.cc:343] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: b0285c75c632457383e81805ab036aaf
I20260618 13:59:39.388636 28483 leader_election.cc:304] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.388890 28799 tablet_bootstrap.cc:492] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:39.389091 30338 raft_consensus.cc:2749] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:39.389200 28799 ts_tablet_manager.cc:1403] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.042s user 0.003s sys 0.000s
I20260618 13:59:39.390563 28797 consensus_queue.cc:260] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:39.391260 28797 ts_tablet_manager.cc:1434] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.016s user 0.002s sys 0.000s
I20260618 13:59:39.391158 28799 raft_consensus.cc:359] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.391331 28799 raft_consensus.cc:385] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.391381 28799 raft_consensus.cc:740] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.391532 28797 tablet_bootstrap.cc:492] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:39.391628 28799 consensus_queue.cc:260] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.392171 28799 ts_tablet_manager.cc:1434] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:39.392397 28798 tablet_bootstrap.cc:492] T 56650c9e20e24a28821cca4279bfde58 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:39.392405 28799 tablet_bootstrap.cc:492] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:39.394013 28798 tablet_bootstrap.cc:654] T 56650c9e20e24a28821cca4279bfde58 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.399971 30151 raft_consensus.cc:493] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.400090 30151 raft_consensus.cc:515] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.400606 30151 leader_election.cc:290] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:39.401518 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038ea0e9c0b04db8b7ec56b065e3cc51" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:39.401520 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038ea0e9c0b04db8b7ec56b065e3cc51" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:39.402196 28346 leader_election.cc:343] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:39.402379 28346 leader_election.cc:343] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.402493 28346 leader_election.cc:304] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.403205 30151 raft_consensus.cc:2749] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:39.407420 28617 consensus_peers.cc:597] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9c17e76ae9b44207998a43e3576908c4. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.412176 29746 ts_tablet_manager.cc:1929] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:39.412714 28797 tablet_bootstrap.cc:654] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.415436 29746 log.cc:1199] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/092a9aba9c6e48f5be9cf5d55ba93678
I20260618 13:59:39.416115 29746 ts_tablet_manager.cc:1950] T 092a9aba9c6e48f5be9cf5d55ba93678 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:39.419828 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 123) took 4753 ms. Trace:
I20260618 13:59:39.420205 28799 tablet_bootstrap.cc:654] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.425297 29566 ts_tablet_manager.cc:1929] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:39.422099 28540 raft_consensus.cc:3060] T f54910015987484e95b295352d859315 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.436065 29566 log.cc:1199] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/7d4144b1b5ea4e968672616bc22bead8
I20260618 13:59:39.436645 29566 ts_tablet_manager.cc:1950] T 7d4144b1b5ea4e968672616bc22bead8 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:39.437628 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 95) took 7307 ms. Trace:
I20260618 13:59:39.438097 30059 raft_consensus.cc:493] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.438855 30059 raft_consensus.cc:515] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.439415 30059 leader_election.cc:290] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.440065 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7766014ee9814f01ae7ac3e849ca3d50" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:39.440470 29746 rpcz_store.cc:276] 0618 13:59:34.666655 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.666713 (+ 58us) service_pool.cc:224] Handling call
0618 13:59:39.339862 (+4673149us) ts_tablet_manager.cc:1151] Deleting tablet 092a9aba9c6e48f5be9cf5d55ba93678
0618 13:59:39.339872 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.378559 (+ 38687us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.412113 (+ 33554us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.419809 (+ 7696us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4672907}
W20260618 13:59:39.440651 28618 leader_election.cc:343] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.440806 29566 rpcz_store.cc:276] 0618 13:59:32.130578 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.130621 (+ 43us) service_pool.cc:224] Handling call
0618 13:59:39.372048 (+7241427us) ts_tablet_manager.cc:1151] Deleting tablet 7d4144b1b5ea4e968672616bc22bead8
0618 13:59:39.372059 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.422107 (+ 50048us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.425245 (+ 3138us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.437603 (+ 12358us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7241265}
I20260618 13:59:39.441294 29566 tablet_replica.cc:333] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:39.441216 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 7d4144b1b5ea4e968672616bc22bead8 (table default.Table_6ad6cf4da29143ff9ad3187cfcb95f6e [id=c2704c9f28bf4351a809b58bf4af056f]) successfully deleted
I20260618 13:59:39.428843 28797 tablet_bootstrap.cc:492] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:39.441676 28797 ts_tablet_manager.cc:1403] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.050s user 0.004s sys 0.000s
I20260618 13:59:39.441816 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 092a9aba9c6e48f5be9cf5d55ba93678 (table default.Table_0a13b90182654346acbbc8f47a56436d [id=c31d65adb55f458c9ced14d598464f89]) successfully deleted
I20260618 13:59:39.442620 29566 raft_consensus.cc:2243] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:39.443182 28797 raft_consensus.cc:359] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.443336 28797 raft_consensus.cc:385] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.443389 28797 raft_consensus.cc:740] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.443620 29566 raft_consensus.cc:2272] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:39.444247 29566 ts_tablet_manager.cc:1916] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:39.444581 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7766014ee9814f01ae7ac3e849ca3d50" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:39.444816 28403 raft_consensus.cc:2468] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:39.445328 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:39.445632 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:39.445919 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 2128dbec33db453ba51070767a15b21e (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]). Expected version 2 got 0
I20260618 13:59:39.446300 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 72c04f55890648f0a8184f3783243f7c (table default.Table_773be314d7e741fcb4fb6eb7d954802e [id=0d83fc68255c458587390c58b12e00d7]). Expected version 2 got 0
I20260618 13:59:39.444717 28798 tablet_bootstrap.cc:492] T 56650c9e20e24a28821cca4279bfde58 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:39.446731 28798 ts_tablet_manager.cc:1403] T 56650c9e20e24a28821cca4279bfde58 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.054s user 0.003s sys 0.003s
I20260618 13:59:39.447330 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:39.447546 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:39.447685 30152 raft_consensus.cc:493] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.447754 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:39.447799 30152 raft_consensus.cc:515] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.447952 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:39.448132 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:39.448315 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:39.448357 30152 leader_election.cc:290] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:39.445009 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 124) took 4777 ms. Trace:
I20260618 13:59:39.449788 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c25fa765ec794c34b0aa43761961a8c9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:39.449783 29746 rpcz_store.cc:276] 0618 13:59:34.667231 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.667280 (+ 49us) service_pool.cc:224] Handling call
0618 13:59:39.444871 (+4777591us) ts_tablet_manager.cc:1151] Deleting tablet d502d67826ff4d7a8b0f515e5d8b6d77
0618 13:59:39.444881 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.444992 (+ 111us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4777432}
W20260618 13:59:39.450318 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d502d67826ff4d7a8b0f515e5d8b6d77 because the tablet was not found. No further retry: Not found: Tablet not found: d502d67826ff4d7a8b0f515e5d8b6d77
W20260618 13:59:39.450410 28346 leader_election.cc:343] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.445955 28617 leader_election.cc:304] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:39.449553 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c25fa765ec794c34b0aa43761961a8c9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:39.451108 30059 raft_consensus.cc:2804] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:39.451185 30059 raft_consensus.cc:493] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:39.451247 30059 raft_consensus.cc:3060] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:39.451789 28346 leader_election.cc:343] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.451893 28346 leader_election.cc:304] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.452245 30151 raft_consensus.cc:2749] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.452766 28797 consensus_queue.cc:260] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.453377 28797 ts_tablet_manager.cc:1434] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260618 13:59:39.453698 28797 tablet_bootstrap.cc:492] T c38a2156a55e4e5eb91d101e9a61eecd P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:39.453780 30059 raft_consensus.cc:515] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.454618 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7766014ee9814f01ae7ac3e849ca3d50" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:39.454741 30059 leader_election.cc:290] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:39.455317 28618 leader_election.cc:343] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.455832 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7766014ee9814f01ae7ac3e849ca3d50" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:39.455952 29303 raft_consensus.cc:493] T 340f6bbf18eb412ca3afec237ff7752d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.456095 28407 raft_consensus.cc:3060] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.458127 29746 tablet_replica.cc:333] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:39.458312 29746 raft_consensus.cc:2243] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:39.458489 29746 raft_consensus.cc:2272] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:39.459030 29746 ts_tablet_manager.cc:1916] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:39.459208 28407 raft_consensus.cc:2468] T 7766014ee9814f01ae7ac3e849ca3d50 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:39.447444 28798 raft_consensus.cc:359] T 56650c9e20e24a28821cca4279bfde58 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.459666 28617 leader_election.cc:304] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:39.459743 28798 raft_consensus.cc:385] T 56650c9e20e24a28821cca4279bfde58 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.460000 30059 raft_consensus.cc:2804] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:39.460114 30059 raft_consensus.cc:697] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:39.460328 30059 consensus_queue.cc:237] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.459885 28798 raft_consensus.cc:740] T 56650c9e20e24a28821cca4279bfde58 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.463792 28798 consensus_queue.cc:260] T 56650c9e20e24a28821cca4279bfde58 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.466617 28799 tablet_bootstrap.cc:492] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:39.466773 28799 ts_tablet_manager.cc:1403] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.074s user 0.005s sys 0.005s
I20260618 13:59:39.467484 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:39.467540 28799 raft_consensus.cc:359] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.467675 28799 raft_consensus.cc:385] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.467723 28799 raft_consensus.cc:740] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.468212 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:39.468403 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:39.468845 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:39.469257 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a]). Expected version 1 got 0
I20260618 13:59:39.469244 29400 raft_consensus.cc:493] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.469383 29400 raft_consensus.cc:515] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.469616 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:39.469831 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:39.470023 29400 leader_election.cc:290] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.470115 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:39.470227 28797 tablet_bootstrap.cc:654] T c38a2156a55e4e5eb91d101e9a61eecd P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.470525 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa93f065b4d64e7b85e30f0690fae541" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:39.470685 28407 raft_consensus.cc:2468] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:39.470356 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:39.470788 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa93f065b4d64e7b85e30f0690fae541" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:39.471069 28617 leader_election.cc:304] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:39.471155 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
W20260618 13:59:39.471472 28618 leader_election.cc:343] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.471531 29024 catalog_manager.cc:5696] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:39.471791 29400 raft_consensus.cc:2804] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:39.471871 29400 raft_consensus.cc:493] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:39.471948 29400 raft_consensus.cc:3060] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.472121 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:39.472496 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:39.472903 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:39.473455 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:39.473691 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:39.473884 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:39.474092 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:39.474291 29024 catalog_manager.cc:5696] T 650ed7ee8b134bc4b9f95028c54f99be P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:39.474861 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_c70d6c5b0dc94b33b4c5772edd2b9dca [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 2 got 0
I20260618 13:59:39.474925 29400 raft_consensus.cc:515] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.475168 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:39.475412 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:39.475754 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:39.475930 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa93f065b4d64e7b85e30f0690fae541" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:39.475992 29400 leader_election.cc:290] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.476179 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:39.476419 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa93f065b4d64e7b85e30f0690fae541" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
W20260618 13:59:39.476446 28618 leader_election.cc:343] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.476576 28407 raft_consensus.cc:3060] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.476887 28798 ts_tablet_manager.cc:1434] T 56650c9e20e24a28821cca4279bfde58 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.030s user 0.001s sys 0.001s
I20260618 13:59:39.477592 30165 raft_consensus.cc:493] T 957670718c8c4b8cb87115cc4a2fa861 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.477746 30165 raft_consensus.cc:515] T 957670718c8c4b8cb87115cc4a2fa861 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.478503 30165 leader_election.cc:290] T 957670718c8c4b8cb87115cc4a2fa861 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.478936 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "957670718c8c4b8cb87115cc4a2fa861" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:39.479646 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "957670718c8c4b8cb87115cc4a2fa861" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:39.480273 28483 leader_election.cc:304] T 957670718c8c4b8cb87115cc4a2fa861 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.480330 28407 raft_consensus.cc:2468] T aa93f065b4d64e7b85e30f0690fae541 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:39.480708 28617 leader_election.cc:304] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:39.480749 30165 raft_consensus.cc:2749] T 957670718c8c4b8cb87115cc4a2fa861 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.481105 30027 raft_consensus.cc:2804] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:39.481232 30027 raft_consensus.cc:697] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:39.481477 30027 consensus_queue.cc:237] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.482647 29023 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_97b6bb39b3bf4612be391b9dd78d1646 [id=42a2a24590834abc9537669aaf223a7c] alter complete (version 1)
W20260618 13:59:39.484930 29017 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ba53ad1988064049a61fcca3a85f8779 consensus state has no leader
I20260618 13:59:39.485170 29017 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 105) took 5094 ms. Trace:
I20260618 13:59:39.485302 29017 rpcz_store.cc:276] 0618 13:59:34.390288 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:37.523166 (+3132878us) service_pool.cc:224] Handling call
0618 13:59:37.523652 (+ 486us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:37.523909 (+ 257us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:37.524130 (+ 221us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:37.524225 (+ 95us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:37.835901 (+311676us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:39.485120 (+1649219us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:37.731534 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:37.819705 (+ 88171us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:37.819715 (+ 10us) write_op.cc:435] Acquired schema lock
0618 13:59:37.819719 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:37.819850 (+ 131us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:37.819867 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:37.819874 (+ 7us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:37.819954 (+ 80us) tablet.cc:701] Row locks acquired
0618 13:59:37.819958 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:37.820004 (+ 46us) write_op.cc:270] Start()
0618 13:59:37.820046 (+ 42us) write_op.cc:276] Timestamp: P: 1781791177819975 usec, L: 0
0618 13:59:37.820051 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:37.820317 (+ 266us) log.cc:844] Serialized 405 byte log entry
0618 13:59:37.834546 (+ 14229us) op_driver.cc:464] REPLICATION: finished
0618 13:59:37.834950 (+ 404us) write_op.cc:301] APPLY: starting
0618 13:59:37.834971 (+ 21us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:37.835019 (+ 48us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:37.835053 (+ 34us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:37.835107 (+ 54us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:37.835141 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:37.835147 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:37.835231 (+ 84us) log.cc:844] Serialized 23 byte log entry
0618 13:59:37.835305 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:37.835326 (+ 21us) write_op.cc:454] Released schema lock
0618 13:59:37.835333 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:37.840842 (+ 5509us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:37.840849 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":330,"apply.run_cpu_time_us":456,"apply.run_wall_time_us":456,"mutex_wait_us":19787,"num_ops":1,"prepare.queue_time_us":206720,"prepare.run_cpu_time_us":1034,"prepare.run_wall_time_us":109304,"raft.queue_time_us":20079,"raft.run_cpu_time_us":652,"raft.run_wall_time_us":650,"replication_time_us":14458}]]}
I20260618 13:59:39.485853 29303 raft_consensus.cc:515] T 340f6bbf18eb412ca3afec237ff7752d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:39.486145 29017 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.2:52853 (request call id 121) took 1618 ms (client timeout 1000 ms). Trace:
W20260618 13:59:39.486248 29017 rpcz_store.cc:269] 0618 13:59:37.867428 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.486074 (+1618646us) service_pool.cc:211] Skipping call since client already timed out
0618 13:59:39.486107 (+ 33us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260618 13:59:39.487125 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340f6bbf18eb412ca3afec237ff7752d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:39.487771 28618 leader_election.cc:343] T 340f6bbf18eb412ca3afec237ff7752d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.488317 28798 tablet_bootstrap.cc:492] T 316110bd2d024c7599fa2933d87e559e P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:39.489424 28799 consensus_queue.cc:260] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.489982 28799 ts_tablet_manager.cc:1434] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.023s user 0.000s sys 0.001s
I20260618 13:59:39.490065 28798 tablet_bootstrap.cc:654] T 316110bd2d024c7599fa2933d87e559e P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.497481 28799 tablet_bootstrap.cc:492] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:39.497766 29019 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_f7e1e81727cb4aa8a4e65ddc115b49c0 [id=803d5220ce1a4dcfaf27d93c574be2ac] alter complete (version 2)
I20260618 13:59:39.498955 28799 tablet_bootstrap.cc:654] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.510804 28549 raft_consensus.cc:3060] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.513170 28799 tablet_bootstrap.cc:492] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:39.513338 28799 ts_tablet_manager.cc:1403] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20260618 13:59:39.514362 28799 raft_consensus.cc:359] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.514604 28799 raft_consensus.cc:385] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.514668 28799 raft_consensus.cc:740] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.520920 28799 consensus_queue.cc:260] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.521732 28799 ts_tablet_manager.cc:1434] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260618 13:59:39.522117 28799 tablet_bootstrap.cc:492] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:39.524497 28799 tablet_bootstrap.cc:654] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.526468 30362 raft_consensus.cc:493] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.526597 30362 raft_consensus.cc:515] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.527194 30362 leader_election.cc:290] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:39.529163 28617 leader_election.cc:343] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 55ae29c3b7ed44cfa659130745037aa6
I20260618 13:59:39.529728 30362 consensus_queue.cc:1048] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:39.530879 30362 raft_consensus.cc:493] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.530992 30362 raft_consensus.cc:515] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:39.531040 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55ae29c3b7ed44cfa659130745037aa6" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:39.531522 30362 leader_election.cc:290] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:39.532470 28617 leader_election.cc:343] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 97a9f949999446f09359629fe483bf67
W20260618 13:59:39.532518 28618 leader_election.cc:343] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.532615 28618 leader_election.cc:304] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:39.533082 29644 raft_consensus.cc:2749] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.533594 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a9f949999446f09359629fe483bf67" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:39.535017 30151 raft_consensus.cc:493] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.535161 30151 raft_consensus.cc:515] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.535801 30151 leader_election.cc:290] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:39.537217 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8da96fc621c47d4a444b5a39259f85d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:39.537833 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8da96fc621c47d4a444b5a39259f85d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:39.538381 28346 leader_election.cc:343] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:39.538604 28346 leader_election.cc:343] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.538686 28346 leader_election.cc:304] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:39.539266 28618 leader_election.cc:343] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.539381 28618 leader_election.cc:304] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:39.539409 30151 raft_consensus.cc:2749] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.539724 29644 raft_consensus.cc:2749] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.540844 30151 raft_consensus.cc:493] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.540968 30151 raft_consensus.cc:515] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.541640 30151 leader_election.cc:290] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:39.542310 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340f6bbf18eb412ca3afec237ff7752d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:39.542593 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340f6bbf18eb412ca3afec237ff7752d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:39.542781 28682 raft_consensus.cc:2468] T 340f6bbf18eb412ca3afec237ff7752d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:39.543551 28346 leader_election.cc:304] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
W20260618 13:59:39.543852 28346 leader_election.cc:343] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.544169 30151 raft_consensus.cc:2804] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:39.544274 30151 raft_consensus.cc:493] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:39.544356 30151 raft_consensus.cc:3060] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.547640 30151 raft_consensus.cc:515] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.548674 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340f6bbf18eb412ca3afec237ff7752d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:39.548857 28682 raft_consensus.cc:3060] T 340f6bbf18eb412ca3afec237ff7752d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.551323 28730 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:39.551796 28682 raft_consensus.cc:2468] T 340f6bbf18eb412ca3afec237ff7752d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:39.552268 28346 leader_election.cc:304] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:39.552568 30152 raft_consensus.cc:2804] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:39.552700 30152 raft_consensus.cc:697] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:39.552963 30152 consensus_queue.cc:237] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.554327 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340f6bbf18eb412ca3afec237ff7752d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:39.554718 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340f6bbf18eb412ca3afec237ff7752d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
W20260618 13:59:39.554840 28346 leader_election.cc:343] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.555009 29303 leader_election.cc:290] T 340f6bbf18eb412ca3afec237ff7752d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.555274 28617 leader_election.cc:304] T 340f6bbf18eb412ca3afec237ff7752d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
W20260618 13:59:39.555425 28618 consensus_peers.cc:597] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.556080 28797 tablet_bootstrap.cc:492] T c38a2156a55e4e5eb91d101e9a61eecd P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:39.556350 28797 ts_tablet_manager.cc:1403] T c38a2156a55e4e5eb91d101e9a61eecd P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.103s user 0.006s sys 0.001s
I20260618 13:59:39.552729 30151 leader_election.cc:290] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:39.557385 28797 raft_consensus.cc:359] T c38a2156a55e4e5eb91d101e9a61eecd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:39.557554 28797 raft_consensus.cc:385] T c38a2156a55e4e5eb91d101e9a61eecd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.557622 28797 raft_consensus.cc:740] T c38a2156a55e4e5eb91d101e9a61eecd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.557891 28797 consensus_queue.cc:260] T c38a2156a55e4e5eb91d101e9a61eecd P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:39.558168 30027 raft_consensus.cc:2749] T 340f6bbf18eb412ca3afec237ff7752d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.559048 28797 ts_tablet_manager.cc:1434] T c38a2156a55e4e5eb91d101e9a61eecd P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:39.559448 28797 tablet_bootstrap.cc:492] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:39.563349 28797 tablet_bootstrap.cc:654] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.575655 30027 raft_consensus.cc:493] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.575824 30027 raft_consensus.cc:515] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:39.577411 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe1139c409e84070a2f496a85545e8d5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:39.577610 28403 raft_consensus.cc:2468] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:39.577986 30027 leader_election.cc:290] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:39.578456 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe1139c409e84070a2f496a85545e8d5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:39.578536 28617 leader_election.cc:304] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:39.578835 30027 raft_consensus.cc:2804] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:39.578918 30027 raft_consensus.cc:493] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:39.579038 30027 raft_consensus.cc:3060] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:39.579056 28618 leader_election.cc:343] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.582129 30027 raft_consensus.cc:515] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:39.583268 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe1139c409e84070a2f496a85545e8d5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:39.583455 28403 raft_consensus.cc:3060] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.586649 28403 raft_consensus.cc:2468] T fe1139c409e84070a2f496a85545e8d5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:39.587169 28617 leader_election.cc:304] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:39.587425 29644 raft_consensus.cc:2804] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:39.587555 29644 raft_consensus.cc:697] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:39.587661 30027 leader_election.cc:290] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:39.587787 29644 consensus_queue.cc:237] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:39.589236 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe1139c409e84070a2f496a85545e8d5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:39.589747 28618 leader_election.cc:343] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.593345 28799 tablet_bootstrap.cc:492] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:39.593490 28799 ts_tablet_manager.cc:1403] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.071s user 0.005s sys 0.000s
I20260618 13:59:39.597323 29303 raft_consensus.cc:493] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.597445 29303 raft_consensus.cc:515] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.598435 29303 leader_election.cc:290] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.598551 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1968e5ce2e42a5b836c90b0904f158" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:39.598753 28407 raft_consensus.cc:2468] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:39.599198 28617 leader_election.cc:304] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:39.599561 29303 raft_consensus.cc:2804] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:39.599709 29303 raft_consensus.cc:493] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:39.599864 29303 raft_consensus.cc:3060] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.602963 29303 raft_consensus.cc:515] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.603456 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1968e5ce2e42a5b836c90b0904f158" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:39.604101 28990 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3a783590159f4a80964fbbcb4a24eef9 consensus state has no leader
W20260618 13:59:39.604193 28618 leader_election.cc:343] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.604211 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1968e5ce2e42a5b836c90b0904f158" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:39.604372 28407 raft_consensus.cc:3060] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.607641 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1968e5ce2e42a5b836c90b0904f158" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:39.608451 28407 raft_consensus.cc:2468] T fc1968e5ce2e42a5b836c90b0904f158 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
W20260618 13:59:39.608527 28618 leader_election.cc:343] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.608877 28617 leader_election.cc:304] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:39.609166 30027 raft_consensus.cc:2804] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:39.609187 28546 raft_consensus.cc:3060] T 15483938a06e459789f87c6bb456bd15 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.609279 30027 raft_consensus.cc:697] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:39.609536 30027 consensus_queue.cc:237] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.615482 28799 raft_consensus.cc:359] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.616003 28799 raft_consensus.cc:385] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.616128 28799 raft_consensus.cc:740] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.616454 28799 consensus_queue.cc:260] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.615828 30151 raft_consensus.cc:493] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.617415 30151 raft_consensus.cc:515] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.619282 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc295a8e81b64859915a6335f8fb8d8e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:39.619441 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc295a8e81b64859915a6335f8fb8d8e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:39.619894 28346 leader_election.cc:343] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:39.620088 28346 leader_election.cc:343] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.620044 30027 consensus_queue.cc:1048] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:39.620173 28346 leader_election.cc:304] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.620465 30262 raft_consensus.cc:2749] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.620539 30151 leader_election.cc:290] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:39.628046 28799 ts_tablet_manager.cc:1434] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.034s user 0.001s sys 0.001s
I20260618 13:59:39.628363 28799 tablet_bootstrap.cc:492] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:39.630721 29303 leader_election.cc:290] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.632169 28797 tablet_bootstrap.cc:492] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:39.632321 28797 ts_tablet_manager.cc:1403] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.073s user 0.016s sys 0.015s
I20260618 13:59:39.633361 28797 raft_consensus.cc:359] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.633632 28797 raft_consensus.cc:385] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.633716 28797 raft_consensus.cc:740] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.633844 28799 tablet_bootstrap.cc:654] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.633993 28797 consensus_queue.cc:260] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.634724 28797 ts_tablet_manager.cc:1434] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260618 13:59:39.635123 28797 tablet_bootstrap.cc:492] T 3a783590159f4a80964fbbcb4a24eef9 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:39.637012 28797 tablet_bootstrap.cc:654] T 3a783590159f4a80964fbbcb4a24eef9 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.640492 30027 consensus_queue.cc:1048] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:39.641556 30027 consensus_queue.cc:1048] T 7f83bb8c3c504d5f9affbaa8be3f1508 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:39.646438 28618 consensus_peers.cc:597] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:39.656178 28618 consensus_peers.cc:597] T 80e99a85e83c46afa367e59e5ddb7dc7 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.656351 30059 raft_consensus.cc:493] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.656497 30059 raft_consensus.cc:515] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.657171 30059 leader_election.cc:290] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.657931 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "438663f365e34fab87e34f434920cecf" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:39.658507 28618 leader_election.cc:343] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.658682 30262 raft_consensus.cc:493] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.658890 30262 raft_consensus.cc:515] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.659698 30262 leader_election.cc:290] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:39.660099 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5115a31a38db48c1a6474917aff97c8e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:39.660503 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5115a31a38db48c1a6474917aff97c8e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:39.660633 28346 leader_election.cc:343] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:39.661252 28346 leader_election.cc:343] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.661387 28346 leader_election.cc:304] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.661934 30151 raft_consensus.cc:2749] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:39.662807 28617 consensus_peers.cc:597] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 46bfe8cc2078414d84e65e22ed4ef8a2. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.663455 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "438663f365e34fab87e34f434920cecf" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:39.663779 28403 raft_consensus.cc:2468] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:39.664243 28617 leader_election.cc:304] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:39.664662 30027 raft_consensus.cc:2804] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:39.664811 30027 raft_consensus.cc:493] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:39.664948 30027 raft_consensus.cc:3060] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.665741 29566 ts_tablet_manager.cc:1929] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:39.665908 29566 log.cc:1199] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/c7f42997416841a7b2c1ad347d2e9282
I20260618 13:59:39.666373 29566 ts_tablet_manager.cc:1950] T c7f42997416841a7b2c1ad347d2e9282 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:39.666666 29303 raft_consensus.cc:493] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.666862 29303 raft_consensus.cc:515] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.667490 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 96) took 7536 ms. Trace:
I20260618 13:59:39.667622 29566 rpcz_store.cc:276] 0618 13:59:32.131397 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.131440 (+ 43us) service_pool.cc:224] Handling call
0618 13:59:39.441249 (+7309809us) ts_tablet_manager.cc:1151] Deleting tablet c7f42997416841a7b2c1ad347d2e9282
0618 13:59:39.441259 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.443168 (+ 1909us) spinlock_profiling.cc:243] Waited 1.14 ms on lock 0x6160000c3a54. stack: 00007f39a472f496 00007f39a472f250 00007f39a375a3ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007f39b07d57e0 00007f39b1b21933 00007f39b46e4ae7 00007f39b4717bb4 00007f39b46f85d0 00007f39b46f835d 0000000000aa681c 00007f39a47759ff 00007f39a4778d1b 00007f39a4778b98
0618 13:59:39.480049 (+ 36881us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.665649 (+185600us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.667464 (+ 1815us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2619776,"tablet-delete.queue_time_us":7309625}
I20260618 13:59:39.668531 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet c7f42997416841a7b2c1ad347d2e9282 (table default.Table_0a13b90182654346acbbc8f47a56436d [id=c31d65adb55f458c9ced14d598464f89]) successfully deleted
I20260618 13:59:39.669005 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc295a8e81b64859915a6335f8fb8d8e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:39.669811 28618 leader_election.cc:343] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.670089 29566 tablet_replica.cc:333] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:39.670480 29566 raft_consensus.cc:2243] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:39.671378 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc295a8e81b64859915a6335f8fb8d8e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:39.671576 28403 raft_consensus.cc:2468] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:39.695394 29746 ts_tablet_manager.cc:1929] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:39.695631 29746 log.cc:1199] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/fe750fc5d998448caeab0a6c808dfba7
I20260618 13:59:39.696091 29746 ts_tablet_manager.cc:1950] T fe750fc5d998448caeab0a6c808dfba7 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:39.696918 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 125) took 5029 ms. Trace:
I20260618 13:59:39.697074 29746 rpcz_store.cc:276] 0618 13:59:34.667638 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.667682 (+ 44us) service_pool.cc:224] Handling call
0618 13:59:39.458080 (+4790398us) ts_tablet_manager.cc:1151] Deleting tablet fe750fc5d998448caeab0a6c808dfba7
0618 13:59:39.458088 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.496107 (+ 38019us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.695325 (+199218us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.696899 (+ 1574us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4790236}
I20260618 13:59:39.697672 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet fe750fc5d998448caeab0a6c808dfba7 (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]) successfully deleted
I20260618 13:59:39.698093 29746 tablet_replica.cc:333] T b16f7d3c7837498e996bba1818110642 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:39.698350 29746 raft_consensus.cc:2243] T b16f7d3c7837498e996bba1818110642 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:39.708431 28799 tablet_bootstrap.cc:492] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:39.708698 28799 ts_tablet_manager.cc:1403] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.080s user 0.004s sys 0.023s
I20260618 13:59:39.709501 28799 raft_consensus.cc:359] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.709867 28799 raft_consensus.cc:385] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.710001 28799 raft_consensus.cc:740] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.710834 28799 consensus_queue.cc:260] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.711548 28799 ts_tablet_manager.cc:1434] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:39.711927 28799 tablet_bootstrap.cc:492] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:39.670893 29303 leader_election.cc:290] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.672098 28617 leader_election.cc:304] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:39.719295 29566 raft_consensus.cc:2272] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:39.719466 28617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 47384.8us
I20260618 13:59:39.719692 30274 raft_consensus.cc:2804] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:39.719774 30274 raft_consensus.cc:493] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:39.719838 30274 raft_consensus.cc:3060] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.674633 30027 raft_consensus.cc:515] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.719964 29566 ts_tablet_manager.cc:1916] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:39.724997 29746 raft_consensus.cc:2272] T b16f7d3c7837498e996bba1818110642 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:39.725579 29746 ts_tablet_manager.cc:1916] T b16f7d3c7837498e996bba1818110642 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:39.699244 28798 tablet_bootstrap.cc:492] T 316110bd2d024c7599fa2933d87e559e P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:39.702723 30165 raft_consensus.cc:493] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.726541 30165 raft_consensus.cc:515] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.727222 30165 leader_election.cc:290] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.727823 28798 ts_tablet_manager.cc:1403] T 316110bd2d024c7599fa2933d87e559e P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.240s user 0.004s sys 0.001s
W20260618 13:59:39.728629 28483 leader_election.cc:343] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 7aa9f65d8a5c46fdb745ee17af946dce
W20260618 13:59:39.728752 28484 leader_election.cc:343] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 7aa9f65d8a5c46fdb745ee17af946dce
I20260618 13:59:39.728906 28484 leader_election.cc:304] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.729149 30165 raft_consensus.cc:2749] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.729012 28798 raft_consensus.cc:359] T 316110bd2d024c7599fa2933d87e559e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.729380 28798 raft_consensus.cc:385] T 316110bd2d024c7599fa2933d87e559e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.729451 28798 raft_consensus.cc:740] T 316110bd2d024c7599fa2933d87e559e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.729678 28798 consensus_queue.cc:260] T 316110bd2d024c7599fa2933d87e559e P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.730294 28798 ts_tablet_manager.cc:1434] T 316110bd2d024c7599fa2933d87e559e P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 13:59:39.730621 28798 tablet_bootstrap.cc:492] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:39.724735 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "438663f365e34fab87e34f434920cecf" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:39.723342 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "438663f365e34fab87e34f434920cecf" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:39.736964 28403 raft_consensus.cc:3060] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.739801 28403 raft_consensus.cc:2468] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:39.726332 30152 raft_consensus.cc:493] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.741998 30152 raft_consensus.cc:515] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:39.744326 30152 leader_election.cc:290] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:39.720412 30027 leader_election.cc:290] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.722246 30274 raft_consensus.cc:515] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:39.734474 28618 leader_election.cc:343] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.747248 30274 leader_election.cc:290] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.747357 28617 leader_election.cc:304] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:39.747833 30274 raft_consensus.cc:2804] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:39.744086 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c38a2156a55e4e5eb91d101e9a61eecd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:39.748044 28617 consensus_peers.cc:597] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 35d8dd2c388e43a0be380aa953ec2c82. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.748071 28682 raft_consensus.cc:2468] T c38a2156a55e4e5eb91d101e9a61eecd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:39.748970 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc295a8e81b64859915a6335f8fb8d8e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:39.749164 28403 raft_consensus.cc:3060] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.749820 28346 leader_election.cc:304] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:39.750269 30274 raft_consensus.cc:697] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:39.750589 30274 consensus_queue.cc:237] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.751750 30152 raft_consensus.cc:2804] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:39.751860 30152 raft_consensus.cc:493] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:39.751945 30152 raft_consensus.cc:3060] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.752624 28403 raft_consensus.cc:2468] T fc295a8e81b64859915a6335f8fb8d8e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:39.753830 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc295a8e81b64859915a6335f8fb8d8e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:39.754539 28618 leader_election.cc:343] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.754895 28617 leader_election.cc:304] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:39.755359 30027 raft_consensus.cc:2804] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:39.755486 30027 raft_consensus.cc:697] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:39.755757 30027 consensus_queue.cc:237] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.756013 30152 raft_consensus.cc:515] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:39.766987 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c38a2156a55e4e5eb91d101e9a61eecd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:39.767946 28346 leader_election.cc:343] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.771162 28799 tablet_bootstrap.cc:654] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.772321 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c38a2156a55e4e5eb91d101e9a61eecd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:39.772960 28346 leader_election.cc:343] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.774655 28798 tablet_bootstrap.cc:654] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.775571 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c38a2156a55e4e5eb91d101e9a61eecd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:39.775807 28682 raft_consensus.cc:3060] T c38a2156a55e4e5eb91d101e9a61eecd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.779490 28682 raft_consensus.cc:2468] T c38a2156a55e4e5eb91d101e9a61eecd P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:39.780055 28346 leader_election.cc:304] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:39.780360 30151 raft_consensus.cc:2804] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:39.780480 30151 raft_consensus.cc:697] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:39.780727 30151 consensus_queue.cc:237] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:39.781832 30152 leader_election.cc:290] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:39.782073 30262 raft_consensus.cc:493] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.782207 30262 raft_consensus.cc:515] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.783550 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a71e334ed1400d97c965b49b677a81" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:39.784135 28346 leader_election.cc:343] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.784548 28798 tablet_bootstrap.cc:492] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:39.784678 28798 ts_tablet_manager.cc:1403] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.054s user 0.003s sys 0.000s
I20260618 13:59:39.785379 28798 raft_consensus.cc:359] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.785535 28798 raft_consensus.cc:385] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.785604 28798 raft_consensus.cc:740] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.785770 29746 ts_tablet_manager.cc:1929] T b16f7d3c7837498e996bba1818110642 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:39.787158 30262 leader_election.cc:290] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:39.787617 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a71e334ed1400d97c965b49b677a81" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:39.788272 28346 leader_election.cc:343] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.788399 28346 leader_election.cc:304] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.788941 30262 raft_consensus.cc:2749] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.795184 28383 tablet_service.cc:1511] Processing CreateTablet for tablet c9382a86a2f645b4857aa04e459209d9 (DEFAULT_TABLE table=default.Table_e33476def9e34806972e63944bfc2998 [id=dc8204244d874b67b8a15433a645cc51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:39.795704 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9382a86a2f645b4857aa04e459209d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:39.798873 28653 tablet_service.cc:1511] Processing CreateTablet for tablet c9382a86a2f645b4857aa04e459209d9 (DEFAULT_TABLE table=default.Table_e33476def9e34806972e63944bfc2998 [id=dc8204244d874b67b8a15433a645cc51]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:39.803903 28346 consensus_peers.cc:597] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.806533 29566 ts_tablet_manager.cc:1929] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:39.807845 29566 log.cc:1199] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/75b9c0754beb4e0e8ed3a7c1e9f368a9
I20260618 13:59:39.808357 29566 ts_tablet_manager.cc:1950] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:39.785879 28798 consensus_queue.cc:260] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.795441 28529 tablet_service.cc:1511] Processing CreateTablet for tablet c9382a86a2f645b4857aa04e459209d9 (DEFAULT_TABLE table=default.Table_e33476def9e34806972e63944bfc2998 [id=dc8204244d874b67b8a15433a645cc51]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:39.805744 28484 consensus_peers.cc:597] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 -> Peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Couldn't send request to peer d6a169b16ced46ca9cfbc631709bb257. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.810861 28529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9382a86a2f645b4857aa04e459209d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:39.810892 28798 ts_tablet_manager.cc:1434] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.026s user 0.002s sys 0.000s
I20260618 13:59:39.811168 28798 tablet_bootstrap.cc:492] T 30f4bd3f86484caab48c1f0806299460 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:39.811313 28546 raft_consensus.cc:3060] T 957670718c8c4b8cb87115cc4a2fa861 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.812500 28798 tablet_bootstrap.cc:654] T 30f4bd3f86484caab48c1f0806299460 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.814407 30262 raft_consensus.cc:493] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.814565 30262 raft_consensus.cc:515] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.815140 30262 leader_election.cc:290] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:39.816504 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac9f580e35af4d4080d313b0283c0b7e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:39.817344 28346 leader_election.cc:343] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.821075 29303 raft_consensus.cc:493] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.821244 29303 raft_consensus.cc:515] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.821962 29303 leader_election.cc:290] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.822640 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b509f940a02545019c526e45e8372c97" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:39.822880 28403 raft_consensus.cc:2468] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:39.823457 28617 leader_election.cc:304] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:39.824254 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b509f940a02545019c526e45e8372c97" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:39.824587 30274 raft_consensus.cc:2804] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:39.824676 30274 raft_consensus.cc:493] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:39.824713 29746 log.cc:1199] T b16f7d3c7837498e996bba1818110642 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/b16f7d3c7837498e996bba1818110642
I20260618 13:59:39.824739 30274 raft_consensus.cc:3060] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:39.824829 28618 leader_election.cc:343] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.825101 29746 ts_tablet_manager.cc:1950] T b16f7d3c7837498e996bba1818110642 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:39.827981 30274 raft_consensus.cc:515] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.829525 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b509f940a02545019c526e45e8372c97" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:39.829600 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b509f940a02545019c526e45e8372c97" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:39.829691 28403 raft_consensus.cc:3060] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:39.830701 28798 tablet_bootstrap.cc:492] T 30f4bd3f86484caab48c1f0806299460 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:39.830823 28798 ts_tablet_manager.cc:1403] T 30f4bd3f86484caab48c1f0806299460 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.007s
W20260618 13:59:39.831017 28618 leader_election.cc:343] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.832340 28403 raft_consensus.cc:2468] T b509f940a02545019c526e45e8372c97 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:39.834903 28798 raft_consensus.cc:359] T 30f4bd3f86484caab48c1f0806299460 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.835059 28798 raft_consensus.cc:385] T 30f4bd3f86484caab48c1f0806299460 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.835115 28798 raft_consensus.cc:740] T 30f4bd3f86484caab48c1f0806299460 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.835331 28798 consensus_queue.cc:260] T 30f4bd3f86484caab48c1f0806299460 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.835772 28798 ts_tablet_manager.cc:1434] T 30f4bd3f86484caab48c1f0806299460 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260618 13:59:39.836025 28798 tablet_bootstrap.cc:492] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:39.837517 28798 tablet_bootstrap.cc:654] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.839331 28617 leader_election.cc:304] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:39.839434 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac9f580e35af4d4080d313b0283c0b7e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:39.839625 30027 raft_consensus.cc:2804] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:39.839720 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9382a86a2f645b4857aa04e459209d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:39.839331 30274 leader_election.cc:290] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.840364 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 98) took 7706 ms. Trace:
I20260618 13:59:39.840447 29566 rpcz_store.cc:276] 0618 13:59:32.133728 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.133797 (+ 69us) service_pool.cc:224] Handling call
0618 13:59:39.670017 (+7536220us) ts_tablet_manager.cc:1151] Deleting tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9
0618 13:59:39.670034 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.761886 (+ 91852us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.806468 (+ 44582us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.840345 (+ 33877us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7535994}
I20260618 13:59:39.840790 29566 tablet_replica.cc:333] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:39.840970 29566 raft_consensus.cc:2243] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:39.841267 29566 raft_consensus.cc:2272] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:39.841361 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 128) took 4944 ms. Trace:
I20260618 13:59:39.841436 29746 rpcz_store.cc:276] 0618 13:59:34.896573 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.896618 (+ 45us) service_pool.cc:224] Handling call
0618 13:59:39.698039 (+4801421us) ts_tablet_manager.cc:1151] Deleting tablet b16f7d3c7837498e996bba1818110642
0618 13:59:39.698051 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.782988 (+ 84937us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.785721 (+ 2733us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.841340 (+ 55619us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4801254}
I20260618 13:59:39.841840 29566 ts_tablet_manager.cc:1916] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:39.841931 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet b16f7d3c7837498e996bba1818110642 (table default.Table_dac9c04f74664cce9c065edc3e491bf6 [id=5fe80671a240424da5ef1d7958caba2a]) successfully deleted
I20260618 13:59:39.842248 29746 tablet_replica.cc:333] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:39.842447 29746 raft_consensus.cc:2243] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:39.839727 30027 raft_consensus.cc:697] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:39.842592 29746 raft_consensus.cc:2272] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:39.842710 30027 consensus_queue.cc:237] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:39.842940 28346 leader_election.cc:343] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.843060 28346 leader_election.cc:304] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.843146 29746 ts_tablet_manager.cc:1916] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:39.843330 30262 raft_consensus.cc:2749] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.844339 30262 consensus_queue.cc:1048] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:39.847906 28483 consensus_peers.cc:597] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac58e939730c4af681c2f08bcfbb6602. This is attempt 6: this message will repeat every 5th retry.
I20260618 13:59:39.848170 28799 tablet_bootstrap.cc:492] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:39.848310 28799 ts_tablet_manager.cc:1403] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.136s user 0.000s sys 0.007s
I20260618 13:59:39.849015 28799 raft_consensus.cc:359] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.849097 30262 raft_consensus.cc:493] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.849315 30262 raft_consensus.cc:515] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.849993 30262 leader_election.cc:290] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:39.850946 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea86899d2c047d997cebe80551e83c9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:39.851646 28346 leader_election.cc:343] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.851929 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea86899d2c047d997cebe80551e83c9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:39.849198 28799 raft_consensus.cc:385] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.853785 28799 raft_consensus.cc:740] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.853998 28799 consensus_queue.cc:260] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:39.854059 28346 leader_election.cc:343] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.854183 28346 leader_election.cc:304] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.854566 30262 raft_consensus.cc:2749] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.855746 28799 ts_tablet_manager.cc:1434] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260618 13:59:39.856403 28799 tablet_bootstrap.cc:492] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:39.857959 28799 tablet_bootstrap.cc:654] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.860549 28798 tablet_bootstrap.cc:492] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:39.860682 28798 ts_tablet_manager.cc:1403] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.001s
I20260618 13:59:39.861408 28798 raft_consensus.cc:359] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.861572 28798 raft_consensus.cc:385] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.861620 28798 raft_consensus.cc:740] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.861809 28798 consensus_queue.cc:260] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.862265 28798 ts_tablet_manager.cc:1434] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
W20260618 13:59:39.872833 28483 consensus_peers.cc:597] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 75b9c0754beb4e0e8ed3a7c1e9f368a9. This is attempt 6: this message will repeat every 5th retry.
I20260618 13:59:39.873582 30165 raft_consensus.cc:493] T 30f4bd3f86484caab48c1f0806299460 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.873745 30165 raft_consensus.cc:515] T 30f4bd3f86484caab48c1f0806299460 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.874480 30165 leader_election.cc:290] T 30f4bd3f86484caab48c1f0806299460 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:39.875108 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30f4bd3f86484caab48c1f0806299460" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:39.876120 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30f4bd3f86484caab48c1f0806299460" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:39.877538 29566 ts_tablet_manager.cc:1929] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:39.878403 29017 catalog_manager.cc:5696] T fa24afe9db46408cace50f642f879f4b P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:39.879288 29017 catalog_manager.cc:5696] T 9fa1a27fc9304e18b96cad9f923779f2 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:39.879890 29017 catalog_manager.cc:5696] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:39.880323 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:39.880656 29566 log.cc:1199] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/0fae928833e44d2ba860d6184d4c0142
I20260618 13:59:39.881439 29566 ts_tablet_manager.cc:1950] T 0fae928833e44d2ba860d6184d4c0142 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:39.881304 29017 catalog_manager.cc:5696] T 15483938a06e459789f87c6bb456bd15 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:39.881999 29017 catalog_manager.cc:5696] T 30f4bd3f86484caab48c1f0806299460 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:39.882287 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 99) took 7747 ms. Trace:
I20260618 13:59:39.882381 29566 rpcz_store.cc:276] 0618 13:59:32.134766 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.134821 (+ 55us) service_pool.cc:224] Handling call
0618 13:59:39.840732 (+7705911us) ts_tablet_manager.cc:1151] Deleting tablet 0fae928833e44d2ba860d6184d4c0142
0618 13:59:39.840743 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.857305 (+ 16562us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.877458 (+ 20153us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.882264 (+ 4806us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7705677}
I20260618 13:59:39.882961 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 0fae928833e44d2ba860d6184d4c0142 (table default.Table_9e1e848ca5c94952ab3928c5feafe2f1 [id=14652783e7f54ebcb5d7ba01d1855118]) successfully deleted
I20260618 13:59:39.883303 28484 leader_election.cc:304] T 30f4bd3f86484caab48c1f0806299460 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.883651 30165 raft_consensus.cc:2749] T 30f4bd3f86484caab48c1f0806299460 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.885010 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:39.885181 29746 ts_tablet_manager.cc:1929] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:39.885361 29746 log.cc:1199] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/bc0adedecee4405e86b0a37b9e62e352
I20260618 13:59:39.885501 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:39.885752 29746 ts_tablet_manager.cc:1950] T bc0adedecee4405e86b0a37b9e62e352 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:39.885766 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:39.886020 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:39.886271 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:39.886579 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 129) took 4989 ms. Trace:
I20260618 13:59:39.886624 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet ba53ad1988064049a61fcca3a85f8779 (table default.Table_b7f097c1abe1410a800ce5aa25771740 [id=6c30b030a60b47e2be48dfe6e9cfdfdb]). Expected version 1 got 0
I20260618 13:59:39.887003 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:39.887439 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:39.888125 29017 catalog_manager.cc:5696] T 4e4c6537144d425f81fd1fafcd034b77 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:39.889043 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:39.889451 28458 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:39.889668 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:39.890741 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:39.891145 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:39.891450 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:39.891716 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:39.886875 28798 tablet_bootstrap.cc:492] T 657e55fa53c447babfe91b1582806e6b P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:39.892298 29017 catalog_manager.cc:5696] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:39.892670 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:39.893391 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:39.893725 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:39.894225 28798 tablet_bootstrap.cc:654] T 657e55fa53c447babfe91b1582806e6b P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.894227 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 2128dbec33db453ba51070767a15b21e (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]). Expected version 2 got 0
I20260618 13:59:39.895901 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:39.895952 28799 tablet_bootstrap.cc:492] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:39.896077 28799 ts_tablet_manager.cc:1403] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.000s
I20260618 13:59:39.896750 28799 raft_consensus.cc:359] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.896914 28799 raft_consensus.cc:385] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.896963 28799 raft_consensus.cc:740] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.897176 28799 consensus_queue.cc:260] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.897318 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:39.897771 28799 ts_tablet_manager.cc:1434] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:39.897876 29746 rpcz_store.cc:276] 0618 13:59:34.896659 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.896718 (+ 59us) service_pool.cc:224] Handling call
0618 13:59:39.842192 (+4945474us) ts_tablet_manager.cc:1151] Deleting tablet bc0adedecee4405e86b0a37b9e62e352
0618 13:59:39.842202 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.876007 (+ 33805us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.885125 (+ 9118us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.886561 (+ 1436us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4945327}
I20260618 13:59:39.898495 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet bc0adedecee4405e86b0a37b9e62e352 (table default.Table_cc20b3f971bb46969baf1fc71e217f2c [id=0a955d9a7fec4cd7985ac56e210fe8a6]) successfully deleted
I20260618 13:59:39.898834 29746 tablet_replica.cc:333] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:39.899052 29746 raft_consensus.cc:2243] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:39.899242 29746 raft_consensus.cc:2272] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:39.899650 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:39.899799 29746 ts_tablet_manager.cc:1916] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:39.899969 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_c70d6c5b0dc94b33b4c5772edd2b9dca [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 2 got 0
I20260618 13:59:39.900282 29017 catalog_manager.cc:5696] T f9f7359c4a144ca89fe3e51d1365a425 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:39.900920 29017 catalog_manager.cc:5696] T 41c92a594cd64982a8972e4a019d1cd8 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:39.901469 28799 tablet_bootstrap.cc:492] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:39.901726 28483 consensus_peers.cc:597] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3dd0eaa64271489d92aa380df877f257. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:39.902536 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:39.903623 28799 tablet_bootstrap.cc:654] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.880373 28797 tablet_bootstrap.cc:492] T 3a783590159f4a80964fbbcb4a24eef9 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:39.904989 28797 ts_tablet_manager.cc:1403] T 3a783590159f4a80964fbbcb4a24eef9 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.270s user 0.020s sys 0.039s
I20260618 13:59:39.902967 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 72c04f55890648f0a8184f3783243f7c (table default.Table_773be314d7e741fcb4fb6eb7d954802e [id=0d83fc68255c458587390c58b12e00d7]). Expected version 2 got 0
I20260618 13:59:39.909720 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:39.910959 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a]). Expected version 1 got 0
I20260618 13:59:39.911371 29017 catalog_manager.cc:5696] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:39.912159 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:39.912632 29017 catalog_manager.cc:5696] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:39.913170 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:39.913677 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:39.914167 28798 tablet_bootstrap.cc:492] T 657e55fa53c447babfe91b1582806e6b P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:39.914291 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:39.914369 28798 ts_tablet_manager.cc:1403] T 657e55fa53c447babfe91b1582806e6b P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.003s
I20260618 13:59:39.914914 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:39.915139 28798 raft_consensus.cc:359] T 657e55fa53c447babfe91b1582806e6b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:39.915395 28798 raft_consensus.cc:385] T 657e55fa53c447babfe91b1582806e6b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.915421 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:39.915493 28798 raft_consensus.cc:740] T 657e55fa53c447babfe91b1582806e6b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.915783 28798 consensus_queue.cc:260] T 657e55fa53c447babfe91b1582806e6b P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:39.915985 29023 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15] alter complete (version 2)
I20260618 13:59:39.916352 28798 ts_tablet_manager.cc:1434] T 657e55fa53c447babfe91b1582806e6b P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 13:59:39.916630 28798 tablet_bootstrap.cc:492] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:39.918262 28798 tablet_bootstrap.cc:654] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.926259 28797 raft_consensus.cc:359] T 3a783590159f4a80964fbbcb4a24eef9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.926504 28797 raft_consensus.cc:385] T 3a783590159f4a80964fbbcb4a24eef9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.926554 28797 raft_consensus.cc:740] T 3a783590159f4a80964fbbcb4a24eef9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.926780 28797 consensus_queue.cc:260] T 3a783590159f4a80964fbbcb4a24eef9 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.927732 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet 18d8b609b96a4e81b57f69bff522eeea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.928283 28596 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:39.932935 29566 tablet_replica.cc:333] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:39.933208 29566 raft_consensus.cc:2243] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:39.934557 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a0039a463ec4a9c82a21a3a69b565fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.934836 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b88ed7cf79c4929a8704f66691cbed4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.935019 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet 233040a16b0d4b61b977ca57c1afec2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.935940 28797 ts_tablet_manager.cc:1434] T 3a783590159f4a80964fbbcb4a24eef9 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.031s user 0.002s sys 0.000s
I20260618 13:59:39.936256 28797 tablet_bootstrap.cc:492] T b968a8d4014341ca9c5964263721ac79 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:39.937875 28797 tablet_bootstrap.cc:654] T b968a8d4014341ca9c5964263721ac79 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.938514 29566 raft_consensus.cc:2272] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:39.939046 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet d7a97176f72349cda221baaea61d18ad with delete_type TABLET_DATA_DELETED (replaced by tablet 72fd63e8245d4042b091593881ba11b6) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.939291 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet e025b64380314546af8a1ab90c817825 with delete_type TABLET_DATA_DELETED (replaced by tablet ece7909d056c4ba8a5dca6423f0c1781) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.939486 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet d4667fe48f0f4eff9c6c8baabdab58d4 with delete_type TABLET_DATA_DELETED (replaced by tablet a662bb37fb7d4df49e61e524808b67a8) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.940661 28798 tablet_bootstrap.cc:492] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:39.941241 30151 raft_consensus.cc:493] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.941375 30151 raft_consensus.cc:515] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.941954 30151 leader_election.cc:290] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:39.941113 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet dd2385e6752d49429422fc854539e9c6 with delete_type TABLET_DATA_DELETED (replaced by tablet c38a2156a55e4e5eb91d101e9a61eecd) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.941717 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet 94365c44b5e24acab5df60c67203794a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.943267 28798 ts_tablet_manager.cc:1403] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.003s
I20260618 13:59:39.942334 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet 35d8dd2c388e43a0be380aa953ec2c82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:34 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.942967 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet f89c9954d48e407c96c294c26644e25c with delete_type TABLET_DATA_DELETED (replaced by tablet b0d3d363766546c4a97b0763bad11f71) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.944007 28798 raft_consensus.cc:359] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.945122 28799 tablet_bootstrap.cc:492] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:39.945086 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b2f34bc17b4a27a91d6cafa9a96c48" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:39.945245 28799 ts_tablet_manager.cc:1403] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.044s user 0.000s sys 0.003s
W20260618 13:59:39.946008 28346 leader_election.cc:343] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.945967 28799 raft_consensus.cc:359] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.946205 28799 raft_consensus.cc:385] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.946276 28799 raft_consensus.cc:740] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.946606 28799 consensus_queue.cc:260] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.955740 28799 ts_tablet_manager.cc:1434] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.010s user 0.000s sys 0.002s
I20260618 13:59:39.944198 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet 8fd9fa9675234db2abff694a517c0ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet 538077e459a44ed78c95f2e881b6e0b0) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.956030 28511 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b59255b14df4158afe37cb8890e7415 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.956118 28799 tablet_bootstrap.cc:492] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:39.944878 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a51e830e3014726b1d1c2608973f036 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.945600 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b2f34bc17b4a27a91d6cafa9a96c48" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:39.956786 28346 leader_election.cc:343] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.956941 28346 leader_election.cc:304] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.957341 30151 raft_consensus.cc:2749] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.957613 28798 raft_consensus.cc:385] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.957917 28799 tablet_bootstrap.cc:654] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.958022 28798 raft_consensus.cc:740] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.958315 28798 consensus_queue.cc:260] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.950882 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet dca54111fea047d9ae0fabe376d89382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.951589 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c75278db2234b11bf95f1cf5cc76edd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.954548 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet d6871232860b41d489e34d95eb897241 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.958567 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet d1baf86497d442dbbbddbc3498bebbee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.959329 28798 ts_tablet_manager.cc:1434] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.016s user 0.001s sys 0.000s
I20260618 13:59:39.959657 28798 tablet_bootstrap.cc:492] T 9024952ab8094fcda1329562bbde8ef5 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:39.960372 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet 41107eb1d9164a9c98060003b4027412 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f5d4296952e48a48756e6f27140da87) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.948637 29746 ts_tablet_manager.cc:1929] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:39.960619 29746 log.cc:1199] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/751cd9bd961b4f56a588ed02a05f0212
I20260618 13:59:39.960944 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet fca76774f119435899188313a33dad98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.962235 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet 7aa9f65d8a5c46fdb745ee17af946dce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.962626 29746 ts_tablet_manager.cc:1950] T 751cd9bd961b4f56a588ed02a05f0212 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:39.968683 30151 raft_consensus.cc:493] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.968808 30151 raft_consensus.cc:515] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.969441 30151 leader_election.cc:290] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:39.970108 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8e487fd51f4aa18bf59e5f8e011dcd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:39.970625 28346 leader_election.cc:343] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.971171 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8e487fd51f4aa18bf59e5f8e011dcd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:39.971606 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 130) took 5072 ms. Trace:
W20260618 13:59:39.971649 28346 leader_election.cc:343] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.971767 28346 leader_election.cc:304] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.972070 30151 raft_consensus.cc:2749] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.963582 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet 316f470ea82f42fca48e4e96d49c9574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.964305 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet aef7c1f111634c98af046e73fa9ffdeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.964839 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet eead369de58f40378254697cbbbc6958 with delete_type TABLET_DATA_DELETED (replaced by tablet d464dd402d2044469e364ed37fdfb39b) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.965396 28511 tablet_service.cc:1558] Processing DeleteTablet for tablet 75824a5d2f7e47838b60e6a2b785c6b2 with delete_type TABLET_DATA_DELETED (replaced by tablet d4539f767c054e9e8ebc1f3f56108e7e) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.966107 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet 4c2b924a096b479cad31c29bf7c99172 with delete_type TABLET_DATA_DELETED (replaced by tablet ab521028ac1a4d0da013112d76bb169f) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.966769 28525 tablet_service.cc:1558] Processing DeleteTablet for tablet 962bf304a285412fb2a529db5b994e9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:34 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.967365 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet fddea8fba4da40d7a5780e6750caf5e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.967950 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet 37a6764a5e024903848561c9935b1a43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.968498 28512 tablet_service.cc:1558] Processing DeleteTablet for tablet ba0d05ea44e1473fb4b24f61eab096e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.969048 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet b6dd422c05324c989e29b20ae2afabad with delete_type TABLET_DATA_DELETED (replaced by tablet 97a9f949999446f09359629fe483bf67) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.970506 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet c3d523419ff244f085d6eac1a3231973 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.962988 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet 36b8a46a41e543c58a946b6764618c7d with delete_type TABLET_DATA_DELETED (replaced by tablet f89c9954d48e407c96c294c26644e25c) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.972513 29746 rpcz_store.cc:276] 0618 13:59:34.899464 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.899520 (+ 56us) service_pool.cc:224] Handling call
0618 13:59:39.898788 (+4999268us) ts_tablet_manager.cc:1151] Deleting tablet 751cd9bd961b4f56a588ed02a05f0212
0618 13:59:39.898798 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.922916 (+ 24118us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.948582 (+ 25666us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:39.971589 (+ 23007us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4999121}
I20260618 13:59:39.973258 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 751cd9bd961b4f56a588ed02a05f0212 (table default.Table_74620afb05b24bf9912f3984efd632aa [id=ad88f16ed9e143cda1e892de63a18849]) successfully deleted
I20260618 13:59:39.973345 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet f3ba9186b3ff47d5a2b158fef619686e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.973549 29746 tablet_replica.cc:333] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:39.973783 29746 raft_consensus.cc:2243] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:39.974262 29746 raft_consensus.cc:2272] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:39.973969 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet d9c00003300246ada2a356e3e1cf684b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.974639 30151 raft_consensus.cc:493] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:39.974737 29566 ts_tablet_manager.cc:1916] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:39.974748 30151 raft_consensus.cc:515] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.974884 29746 ts_tablet_manager.cc:1916] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:39.975250 30151 leader_election.cc:290] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:39.975881 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a662bb37fb7d4df49e61e524808b67a8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:39.975909 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a662bb37fb7d4df49e61e524808b67a8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:39.976337 28346 leader_election.cc:343] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:39.976492 28346 leader_election.cc:343] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:39.976574 28346 leader_election.cc:304] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:39.976913 30151 raft_consensus.cc:2749] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:39.979071 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet c4cfca9e76e641a7bc0b58b9ee240f36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.972259 28798 tablet_bootstrap.cc:654] T 9024952ab8094fcda1329562bbde8ef5 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:39.981663 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 516421cd843b44769efa2b555697c065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:39.981755 28799 tablet_bootstrap.cc:492] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:39.981870 28799 ts_tablet_manager.cc:1403] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.003s
I20260618 13:59:39.982028 29020 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_0158779f563348c78b3505f98d43b839 [id=b1d85ea3aa7047d285ab7c86da8213fd] alter complete (version 1)
I20260618 13:59:39.982714 28799 raft_consensus.cc:359] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.982873 28799 raft_consensus.cc:385] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:39.982928 28799 raft_consensus.cc:740] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:39.983166 28799 consensus_queue.cc:260] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:39.987442 28799 ts_tablet_manager.cc:1434] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260618 13:59:39.987818 28799 tablet_bootstrap.cc:492] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:39.989921 28799 tablet_bootstrap.cc:654] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:40.006440 29024 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet 2128dbec33db453ba51070767a15b21e (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 2128dbec33db453ba51070767a15b21e consensus state has no leader
I20260618 13:59:40.007609 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 9c17e76ae9b44207998a43e3576908c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 2128dbec33db453ba51070767a15b21e) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.008065 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a0039a463ec4a9c82a21a3a69b565fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.008904 29746 ts_tablet_manager.cc:1929] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:40.013167 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet b0285c75c632457383e81805ab036aaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.013701 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet d667ea5f029f46d89fcb3be03069b97b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.013710 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet aa8cf927bed9437086efd8e0f21ba26a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.013954 29746 log.cc:1199] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/e95c88e4b0a046a3ad69ee49af47c7c3
I20260618 13:59:40.014273 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet f45556a59c424ca2bc30956566374c89 with delete_type TABLET_DATA_DELETED (replaced by tablet 6903cb63c5b04408b7eaecf0c0231c42) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.014739 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet b6dd422c05324c989e29b20ae2afabad with delete_type TABLET_DATA_DELETED (replaced by tablet 97a9f949999446f09359629fe483bf67) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.015308 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet d7a97176f72349cda221baaea61d18ad with delete_type TABLET_DATA_DELETED (replaced by tablet 72fd63e8245d4042b091593881ba11b6) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.015586 29746 ts_tablet_manager.cc:1950] T e95c88e4b0a046a3ad69ee49af47c7c3 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:40.015769 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet d4667fe48f0f4eff9c6c8baabdab58d4 with delete_type TABLET_DATA_DELETED (replaced by tablet a662bb37fb7d4df49e61e524808b67a8) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.016356 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet 8fd9fa9675234db2abff694a517c0ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet 538077e459a44ed78c95f2e881b6e0b0) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.016963 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet 98b42d23832545ae81a7cdff19e4689f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:33 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.017449 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f5a334d18ba4c06bb358fa86d8af80d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.018028 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet b51d0851efb34eaa9e9e4be6b292ef62 with delete_type TABLET_DATA_DELETED (replaced by tablet f9f7359c4a144ca89fe3e51d1365a425) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.018620 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet fe750fc5d998448caeab0a6c808dfba7 with delete_type TABLET_DATA_DELETED (replaced by tablet 438663f365e34fab87e34f434920cecf) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.019120 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet 55ae29c3b7ed44cfa659130745037aa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:31 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.019374 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 131) took 5117 ms. Trace:
I20260618 13:59:40.019742 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet d1baf86497d442dbbbddbc3498bebbee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.019825 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 7aa9f65d8a5c46fdb745ee17af946dce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.019879 29746 rpcz_store.cc:276] 0618 13:59:34.902111 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.902214 (+ 103us) service_pool.cc:224] Handling call
0618 13:59:39.973507 (+5071293us) ts_tablet_manager.cc:1151] Deleting tablet e95c88e4b0a046a3ad69ee49af47c7c3
0618 13:59:39.973515 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.005089 (+ 31574us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.008822 (+ 3733us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.019346 (+ 10524us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5071161}
I20260618 13:59:40.023129 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet bce7bf74b6d24d0bb43808b8a62cf1b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.023371 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a0691de253b4b008e5d995fd6b876c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.023613 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.023864 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet dd2385e6752d49429422fc854539e9c6 with delete_type TABLET_DATA_DELETED (replaced by tablet c38a2156a55e4e5eb91d101e9a61eecd) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.024008 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet d502d67826ff4d7a8b0f515e5d8b6d77 with delete_type TABLET_DATA_DELETED (replaced by tablet 657e55fa53c447babfe91b1582806e6b) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.024081 29746 tablet_replica.cc:333] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:40.024360 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 751cd9bd961b4f56a588ed02a05f0212 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.024724 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet c557afc3abfa4ef188bf248e89f49d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.025065 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet cd06bb7cffc34412b420d70435fce373 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.025137 30165 consensus_queue.cc:1048] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:40.025439 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e25027f423144b088dad955c7282921 with delete_type TABLET_DATA_DELETED (replaced by tablet a8da96fc621c47d4a444b5a39259f85d) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.025661 29746 raft_consensus.cc:2243] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:40.025847 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet e95c88e4b0a046a3ad69ee49af47c7c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.026335 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 04dac931e62a448d99633878eb70b92f with delete_type TABLET_DATA_DELETED (replaced by tablet 15483938a06e459789f87c6bb456bd15) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.026700 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fae928833e44d2ba860d6184d4c0142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.027077 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet c7f42997416841a7b2c1ad347d2e9282 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:38230
W20260618 13:59:40.027207 29025 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 6903cb63c5b04408b7eaecf0c0231c42 consensus state has no leader
I20260618 13:59:40.027493 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet b16f7d3c7837498e996bba1818110642 with delete_type TABLET_DATA_DELETED (replaced by tablet d1baf86497d442dbbbddbc3498bebbee) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.027855 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 283523e5b22843ba935eabe6283d3a99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.028303 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet aef7c1f111634c98af046e73fa9ffdeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.028661 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 46bfe8cc2078414d84e65e22ed4ef8a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.031195 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f3d2f236fd44ab8af0524ea282cfa94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.031608 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 962bf304a285412fb2a529db5b994e9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:34 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.031982 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c75278db2234b11bf95f1cf5cc76edd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.032092 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f5a334d18ba4c06bb358fa86d8af80d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:40.032325 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 516421cd843b44769efa2b555697c065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:40.033619 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 7d4144b1b5ea4e968672616bc22bead8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:40.033902 29746 raft_consensus.cc:2272] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.034123 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 4c2b924a096b479cad31c29bf7c99172 with delete_type TABLET_DATA_DELETED (replaced by tablet ab521028ac1a4d0da013112d76bb169f) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:40.034646 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet da4d4f03911f4ac099ecc2fa1d8af494 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:40.035149 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet aa8cf927bed9437086efd8e0f21ba26a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:40.035598 29746 ts_tablet_manager.cc:1916] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:40.035832 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet c557afc3abfa4ef188bf248e89f49d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:40.036335 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet c7f42997416841a7b2c1ad347d2e9282 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:40.036808 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet d4667fe48f0f4eff9c6c8baabdab58d4 with delete_type TABLET_DATA_DELETED (replaced by tablet a662bb37fb7d4df49e61e524808b67a8) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:40.037302 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet b16f7d3c7837498e996bba1818110642 with delete_type TABLET_DATA_DELETED (replaced by tablet d1baf86497d442dbbbddbc3498bebbee) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:40.037782 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet bc0adedecee4405e86b0a37b9e62e352 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:40.038266 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 with delete_type TABLET_DATA_DELETED (replaced by tablet cea4bc13eabc4dd98b4f96723498014c) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:40.039040 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fae928833e44d2ba860d6184d4c0142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:40.042451 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 7d4144b1b5ea4e968672616bc22bead8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.042850 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet f89c9954d48e407c96c294c26644e25c with delete_type TABLET_DATA_DELETED (replaced by tablet b0d3d363766546c4a97b0763bad11f71) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.043221 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 092a9aba9c6e48f5be9cf5d55ba93678 with delete_type TABLET_DATA_DELETED (replaced by tablet c7f42997416841a7b2c1ad347d2e9282) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.043644 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 41107eb1d9164a9c98060003b4027412 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f5d4296952e48a48756e6f27140da87) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.044239 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet bc0adedecee4405e86b0a37b9e62e352 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.044656 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 827cc0987bd541759e334fb620f04ed5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.045047 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet e3d8616846c649b6a969b971db0d03db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.045450 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 591465e2b57247a2b224cc8b09225811 with delete_type TABLET_DATA_DELETED (replaced by tablet d4667fe48f0f4eff9c6c8baabdab58d4) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.045799 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet e6de1293572f4e75af3417f281b9a142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.046255 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 75824a5d2f7e47838b60e6a2b785c6b2 with delete_type TABLET_DATA_DELETED (replaced by tablet d4539f767c054e9e8ebc1f3f56108e7e) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.046628 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 02b5cc08ca0c4e719e0414b345211ba5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.046962 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet cea4bc13eabc4dd98b4f96723498014c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.047386 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 6eea4d59935d40aa97dc10ec7469aa56 with delete_type TABLET_DATA_DELETED (replaced by tablet 4bc87770baa34842a25ed97a19354446) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.047763 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 0463fa2e30554abcbfc8f37b702e81d4 with delete_type TABLET_DATA_DELETED (replaced by tablet fe1139c409e84070a2f496a85545e8d5) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.048103 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet be59f988f0e34b9480f859cf88b9c223 with delete_type TABLET_DATA_DELETED (replaced by tablet b968a8d4014341ca9c5964263721ac79) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.048691 28390 tablet_service.cc:1558] Processing DeleteTablet for tablet 6c4a31a4f202473d97b17b5b5dfdb615 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.048686 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet 98446f40214c4ac984fcfe98310005c5 with delete_type TABLET_DATA_DELETED (replaced by tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.049065 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 015494374645469785f1f978eaf0e9a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 2e493f647a344cd581515a77230d765f) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.051580 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 35d8dd2c388e43a0be380aa953ec2c82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:34 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.052102 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet e95c88e4b0a046a3ad69ee49af47c7c3 (table default.Table_68ab7758379b44628a68285f209716ef [id=61ce11eead544e74a31db940c3376f4b]) successfully deleted
I20260618 13:59:40.053777 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet c3d523419ff244f085d6eac1a3231973 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.054119 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet f3ba9186b3ff47d5a2b158fef619686e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.054486 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 7572d17724d249739be1a1cb9468f909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:27 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.054844 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet ab521028ac1a4d0da013112d76bb169f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.055189 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 18d8b609b96a4e81b57f69bff522eeea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.055487 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet ca2ee746c5024cfc9b31c470f9616e7d with delete_type TABLET_DATA_DELETED (replaced by tablet 60b6216474b74a14af6abf977e312b77) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.055794 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet c01c8463055e4526b3c75190066f30b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.056119 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 97a9f949999446f09359629fe483bf67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:29 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.056548 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet e5f7eaa7d218440ca2c35340c5c87814 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:40.064291 28048 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu with pid 29476
I20260618 13:59:40.064750 30151 raft_consensus.cc:493] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.064926 30151 raft_consensus.cc:515] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.065748 30151 leader_election.cc:290] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:40.066766 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a783590159f4a80964fbbcb4a24eef9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.066985 28682 raft_consensus.cc:2468] T 3a783590159f4a80964fbbcb4a24eef9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:40.068401 29566 ts_tablet_manager.cc:1929] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:40.068532 29566 log.cc:1199] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/827cc0987bd541759e334fb620f04ed5
I20260618 13:59:40.068909 29566 ts_tablet_manager.cc:1950] T 827cc0987bd541759e334fb620f04ed5 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:40.069191 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a783590159f4a80964fbbcb4a24eef9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:40.069429 28346 leader_election.cc:304] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:40.069638 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 100) took 7933 ms. Trace:
I20260618 13:59:40.069720 29566 rpcz_store.cc:276] 0618 13:59:32.135904 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.135959 (+ 55us) service_pool.cc:224] Handling call
0618 13:59:39.932870 (+7796911us) ts_tablet_manager.cc:1151] Deleting tablet 827cc0987bd541759e334fb620f04ed5
0618 13:59:39.932885 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:39.977673 (+ 44788us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.068343 (+ 90670us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.069620 (+ 1277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4884,"tablet-delete.queue_time_us":7796686}
I20260618 13:59:40.070086 29566 tablet_replica.cc:333] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:40.070304 29566 raft_consensus.cc:2243] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260618 13:59:40.071995 29024 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.1:48495 (request call id 148) took 2273 ms (client timeout 1000 ms). Trace:
W20260618 13:59:40.072101 29024 rpcz_store.cc:269] 0618 13:59:37.798666 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:37.853607 (+ 54941us) service_pool.cc:224] Handling call
0618 13:59:37.861559 (+ 7952us) spinlock_profiling.cc:243] Waited 3.29 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb467ef08 00007fcbab0ddca6 00007fcbab0dd9d5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:40.071936 (+2210377us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:39.544569 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:39.544732 (+ 163us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:39.544742 (+ 10us) write_op.cc:435] Acquired schema lock
0618 13:59:39.544745 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:39.544942 (+ 197us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:39.544962 (+ 20us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:39.544969 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:39.545111 (+ 142us) tablet.cc:701] Row locks acquired
0618 13:59:39.545116 (+ 5us) write_op.cc:260] PREPARE: finished
0618 13:59:39.545161 (+ 45us) write_op.cc:270] Start()
0618 13:59:39.545204 (+ 43us) write_op.cc:276] Timestamp: P: 1781791179545118 usec, L: 0
0618 13:59:39.545210 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:39.545440 (+ 230us) log.cc:844] Serialized 907 byte log entry
0618 13:59:39.709577 (+164137us) op_driver.cc:464] REPLICATION: finished
0618 13:59:39.741701 (+ 32124us) write_op.cc:301] APPLY: starting
0618 13:59:39.741721 (+ 20us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:39.741767 (+ 46us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:39.741819 (+ 52us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:39.741937 (+ 118us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:39.741971 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:39.741978 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:39.742067 (+ 89us) log.cc:844] Serialized 29 byte log entry
0618 13:59:39.742139 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:39.742159 (+ 20us) write_op.cc:454] Released schema lock
0618 13:59:39.742168 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:39.764656 (+ 22488us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:39.764666 (+ 10us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":255228,"reported_tablets":178,"spinlock_wait_cycles":7556352,"child_traces":[["op",{"apply.queue_time_us":32018,"apply.run_cpu_time_us":596,"apply.run_wall_time_us":22943,"num_ops":2,"prepare.queue_time_us":23304,"prepare.run_cpu_time_us":1103,"prepare.run_wall_time_us":21974,"raft.queue_time_us":358,"raft.run_cpu_time_us":694,"raft.run_wall_time_us":695,"replication_time_us":164327,"thread_start_us":55080,"threads_started":2}]]}
I20260618 13:59:40.073489 30151 raft_consensus.cc:2804] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:40.073586 30151 raft_consensus.cc:493] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:40.073664 30151 raft_consensus.cc:3060] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:40.078588 30151 raft_consensus.cc:515] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.079798 30151 leader_election.cc:290] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:40.080556 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a783590159f4a80964fbbcb4a24eef9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:40.080821 28682 raft_consensus.cc:3060] T 3a783590159f4a80964fbbcb4a24eef9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:40.083568 30152 raft_consensus.cc:493] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.083601 29746 ts_tablet_manager.cc:1929] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:40.083700 29746 log.cc:1199] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/bce7bf74b6d24d0bb43808b8a62cf1b7
I20260618 13:59:40.083684 30152 raft_consensus.cc:515] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.084072 29746 ts_tablet_manager.cc:1950] T bce7bf74b6d24d0bb43808b8a62cf1b7 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:40.084250 30152 leader_election.cc:290] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:40.084785 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 126) took 5188 ms. Trace:
I20260618 13:59:40.084895 29746 rpcz_store.cc:276] 0618 13:59:34.896378 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.910458 (+ 14080us) service_pool.cc:224] Handling call
0618 13:59:40.023961 (+5113503us) ts_tablet_manager.cc:1151] Deleting tablet bce7bf74b6d24d0bb43808b8a62cf1b7
0618 13:59:40.023977 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.042370 (+ 18393us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.083537 (+ 41167us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.084766 (+ 1229us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5113298}
I20260618 13:59:40.085052 28682 raft_consensus.cc:2468] T 3a783590159f4a80964fbbcb4a24eef9 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:40.090144 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a783590159f4a80964fbbcb4a24eef9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:40.091374 28346 leader_election.cc:343] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:40.091778 28346 leader_election.cc:343] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.093132 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5836982752b4209b72a372c1dfeb71c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:40.094072 29746 tablet_replica.cc:333] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:40.094527 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5836982752b4209b72a372c1dfeb71c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.094624 29746 raft_consensus.cc:2243] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:40.094988 29746 raft_consensus.cc:2272] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.095459 28546 raft_consensus.cc:3060] T 56650c9e20e24a28821cca4279bfde58 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:40.095781 29746 ts_tablet_manager.cc:1916] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:40.101349 28346 leader_election.cc:304] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:40.102347 30152 raft_consensus.cc:2804] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:40.102496 30152 raft_consensus.cc:697] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:40.102739 30152 consensus_queue.cc:237] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:40.108839 28346 leader_election.cc:343] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.112156 28549 raft_consensus.cc:3060] T 30f4bd3f86484caab48c1f0806299460 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:40.112443 28346 consensus_peers.cc:597] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:40.114588 28346 leader_election.cc:343] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.115127 28346 leader_election.cc:304] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.115643 30152 raft_consensus.cc:2749] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:40.116832 29746 ts_tablet_manager.cc:1929] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:40.118680 29746 log.cc:1199] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/b0285c75c632457383e81805ab036aaf
I20260618 13:59:40.119275 29746 ts_tablet_manager.cc:1950] T b0285c75c632457383e81805ab036aaf P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:40.122180 30151 consensus_queue.cc:1048] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:40.123193 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 127) took 5226 ms. Trace:
I20260618 13:59:40.123281 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet bce7bf74b6d24d0bb43808b8a62cf1b7 (table default.Table_3cdafadde7c146d38bdb87c358269c35 [id=214e8abbe81c4ff19009a4902bdef469]) successfully deleted
I20260618 13:59:40.123550 29746 rpcz_store.cc:276] 0618 13:59:34.896481 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.911251 (+ 14770us) service_pool.cc:224] Handling call
0618 13:59:40.094023 (+5182772us) ts_tablet_manager.cc:1151] Deleting tablet b0285c75c632457383e81805ab036aaf
0618 13:59:40.094033 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.104338 (+ 10305us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.116771 (+ 12433us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.123170 (+ 6399us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5182614}
I20260618 13:59:40.123852 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 827cc0987bd541759e334fb620f04ed5 (table default.Table_9bfb035b965742abaaf21413be078bf6 [id=9be95b9adc544c0e9675071330a21225]) successfully deleted
I20260618 13:59:40.125087 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet b0285c75c632457383e81805ab036aaf (table default.Table_9cd59394f11d4937887809241a69bba3 [id=75fcc464e22646f7960501f112b3f887]) successfully deleted
I20260618 13:59:40.125507 29746 tablet_replica.cc:333] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:40.125929 29746 raft_consensus.cc:2243] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:40.126353 29746 raft_consensus.cc:2272] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.131280 29746 ts_tablet_manager.cc:1916] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:40.132129 28617 consensus_peers.cc:597] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 864f2602840c40fc8606b3784e671cba. This is attempt 6: this message will repeat every 5th retry.
I20260618 13:59:40.133642 30151 raft_consensus.cc:493] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.133877 30151 raft_consensus.cc:515] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.134548 30151 leader_election.cc:290] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:40.136175 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e493f647a344cd581515a77230d765f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:40.138145 30165 raft_consensus.cc:493] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.138299 30165 raft_consensus.cc:515] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:40.140703 28483 leader_election.cc:343] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: f3ba9186b3ff47d5a2b158fef619686e
I20260618 13:59:40.140966 30165 leader_election.cc:290] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:40.141631 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e493f647a344cd581515a77230d765f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.144469 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ba9186b3ff47d5a2b158fef619686e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.146647 28484 leader_election.cc:304] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.146790 29746 ts_tablet_manager.cc:1929] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:40.146967 30165 raft_consensus.cc:2749] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:40.147378 29746 log.cc:1199] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/d667ea5f029f46d89fcb3be03069b97b
I20260618 13:59:40.152041 29746 ts_tablet_manager.cc:1950] T d667ea5f029f46d89fcb3be03069b97b P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
W20260618 13:59:40.152489 28618 consensus_peers.cc:597] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 75824a5d2f7e47838b60e6a2b785c6b2. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:40.153468 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 132) took 5161 ms. Trace:
I20260618 13:59:40.154400 29746 rpcz_store.cc:276] 0618 13:59:34.992321 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.992538 (+ 217us) service_pool.cc:224] Handling call
0618 13:59:40.125451 (+5132913us) ts_tablet_manager.cc:1151] Deleting tablet d667ea5f029f46d89fcb3be03069b97b
0618 13:59:40.125460 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.135408 (+ 9948us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.146717 (+ 11309us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.153448 (+ 6731us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5072545}
I20260618 13:59:40.155267 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet d667ea5f029f46d89fcb3be03069b97b (table default.Table_f4a03a199e51400ca4c8a5d868ec2025 [id=0ce6f37b68d8435f8f860ba43647bc3b]) successfully deleted
I20260618 13:59:40.155711 29746 tablet_replica.cc:333] T 04dac931e62a448d99633878eb70b92f P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:40.156196 29746 raft_consensus.cc:2243] T 04dac931e62a448d99633878eb70b92f P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:40.156715 29746 raft_consensus.cc:2272] T 04dac931e62a448d99633878eb70b92f P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.157578 29746 ts_tablet_manager.cc:1916] T 04dac931e62a448d99633878eb70b92f P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:40.157560 28346 leader_election.cc:343] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:40.163856 28618 consensus_peers.cc:597] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:40.172093 28797 tablet_bootstrap.cc:492] T b968a8d4014341ca9c5964263721ac79 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:40.172317 28797 ts_tablet_manager.cc:1403] T b968a8d4014341ca9c5964263721ac79 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.236s user 0.002s sys 0.001s
W20260618 13:59:40.172412 28346 leader_election.cc:343] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.172530 28346 leader_election.cc:304] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.173796 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f5a334d18ba4c06bb358fa86d8af80d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:40.174122 28797 raft_consensus.cc:359] T b968a8d4014341ca9c5964263721ac79 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:40.174436 28797 raft_consensus.cc:385] T b968a8d4014341ca9c5964263721ac79 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.174643 30151 raft_consensus.cc:2749] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:40.175817 29025 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.3:42065 (request call id 146) took 2518 ms (client timeout 1000 ms). Trace:
I20260618 13:59:40.174548 28797 raft_consensus.cc:740] T b968a8d4014341ca9c5964263721ac79 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.176422 28797 consensus_queue.cc:260] T b968a8d4014341ca9c5964263721ac79 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:40.176937 28797 ts_tablet_manager.cc:1434] T b968a8d4014341ca9c5964263721ac79 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260618 13:59:40.177099 28798 tablet_bootstrap.cc:492] T 9024952ab8094fcda1329562bbde8ef5 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:40.177232 28797 tablet_bootstrap.cc:492] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:40.178699 28797 tablet_bootstrap.cc:654] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:40.179764 29018 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.1:48495 (request call id 147) took 3467 ms (client timeout 1000 ms). Trace:
I20260618 13:59:40.180138 28798 ts_tablet_manager.cc:1403] T 9024952ab8094fcda1329562bbde8ef5 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.221s user 0.002s sys 0.002s
I20260618 13:59:40.181550 28798 raft_consensus.cc:359] T 9024952ab8094fcda1329562bbde8ef5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.183975 28797 tablet_bootstrap.cc:492] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:40.184120 28797 ts_tablet_manager.cc:1403] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260618 13:59:40.184197 30151 raft_consensus.cc:493] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.184329 30151 raft_consensus.cc:515] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.185014 30151 leader_election.cc:290] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:40.185925 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9ceb3af08fa4618b81cf6f89d7bd0df" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:40.186591 28346 leader_election.cc:343] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.187007 28797 raft_consensus.cc:359] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.187224 28797 raft_consensus.cc:385] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.187278 28797 raft_consensus.cc:740] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.188508 28525 tablet_service.cc:1558] Processing DeleteTablet for tablet fe750fc5d998448caeab0a6c808dfba7 with delete_type TABLET_DATA_DELETED (replaced by tablet 438663f365e34fab87e34f434920cecf) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:40.183769 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet c84189a4b1194c87bbdf81f6e0ed56e4 with delete_type TABLET_DATA_DELETED (replaced by tablet be00bfa9bc1a4b83b18bfad05ba1495e) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:40.184621 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fae928833e44d2ba860d6184d4c0142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:40.185448 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet aa8cf927bed9437086efd8e0f21ba26a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:40.188897 28797 consensus_queue.cc:260] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.188522 28798 raft_consensus.cc:385] T 9024952ab8094fcda1329562bbde8ef5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.189436 28798 raft_consensus.cc:740] T 9024952ab8094fcda1329562bbde8ef5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.189785 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet bce7bf74b6d24d0bb43808b8a62cf1b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:40.189873 28797 ts_tablet_manager.cc:1434] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260618 13:59:40.190135 28797 tablet_bootstrap.cc:492] T f1663f9d92e54f1caf3c7137412e495c P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:40.189673 28798 consensus_queue.cc:260] T 9024952ab8094fcda1329562bbde8ef5 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.190599 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet b0285c75c632457383e81805ab036aaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:40.191429 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet b16f7d3c7837498e996bba1818110642 with delete_type TABLET_DATA_DELETED (replaced by tablet d1baf86497d442dbbbddbc3498bebbee) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:40.192319 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet bc0adedecee4405e86b0a37b9e62e352 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:40.193143 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet 751cd9bd961b4f56a588ed02a05f0212 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:40.193661 28798 ts_tablet_manager.cc:1434] T 9024952ab8094fcda1329562bbde8ef5 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260618 13:59:40.194327 28797 tablet_bootstrap.cc:654] T f1663f9d92e54f1caf3c7137412e495c P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.186168 28512 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:40.194706 28798 tablet_bootstrap.cc:492] T bf5e04eb43ed4cf493c0f6f08e22baeb P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:40.194818 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet d667ea5f029f46d89fcb3be03069b97b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:40.186892 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9ceb3af08fa4618b81cf6f89d7bd0df" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:40.186926 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet b51d0851efb34eaa9e9e4be6b292ef62 with delete_type TABLET_DATA_DELETED (replaced by tablet f9f7359c4a144ca89fe3e51d1365a425) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:40.187696 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet 092a9aba9c6e48f5be9cf5d55ba93678 with delete_type TABLET_DATA_DELETED (replaced by tablet c7f42997416841a7b2c1ad347d2e9282) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:40.193981 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet e95c88e4b0a046a3ad69ee49af47c7c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:57282
W20260618 13:59:40.195604 28346 leader_election.cc:343] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.195777 28346 leader_election.cc:304] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.196278 30151 raft_consensus.cc:2749] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:40.200762 28797 tablet_bootstrap.cc:492] T f1663f9d92e54f1caf3c7137412e495c P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:40.200903 28797 ts_tablet_manager.cc:1403] T f1663f9d92e54f1caf3c7137412e495c P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260618 13:59:40.201663 28797 raft_consensus.cc:359] T f1663f9d92e54f1caf3c7137412e495c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.201820 28797 raft_consensus.cc:385] T f1663f9d92e54f1caf3c7137412e495c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.201870 28797 raft_consensus.cc:740] T f1663f9d92e54f1caf3c7137412e495c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.204075 28798 tablet_bootstrap.cc:654] T bf5e04eb43ed4cf493c0f6f08e22baeb P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.208187 28797 consensus_queue.cc:260] T f1663f9d92e54f1caf3c7137412e495c P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.209261 28797 ts_tablet_manager.cc:1434] T f1663f9d92e54f1caf3c7137412e495c P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260618 13:59:40.209653 28797 tablet_bootstrap.cc:492] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:40.216501 30165 raft_consensus.cc:493] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.216666 30165 raft_consensus.cc:515] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.217303 30165 leader_election.cc:290] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:40.218730 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d046e0421142cd84c5b12d6a434f53" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:40.219419 29400 consensus_queue.cc:1048] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:40.225907 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d046e0421142cd84c5b12d6a434f53" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.227973 28484 leader_election.cc:304] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.228284 30165 raft_consensus.cc:2749] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:40.230749 30165 raft_consensus.cc:493] T 316110bd2d024c7599fa2933d87e559e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.230916 30165 raft_consensus.cc:515] T 316110bd2d024c7599fa2933d87e559e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.231598 30165 leader_election.cc:290] T 316110bd2d024c7599fa2933d87e559e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:40.232466 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316110bd2d024c7599fa2933d87e559e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:40.232712 28797 tablet_bootstrap.cc:654] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:40.195500 29025 rpcz_store.cc:269] 0618 13:59:37.512173 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:37.514781 (+ 2608us) service_pool.cc:224] Handling call
0618 13:59:40.030148 (+2515367us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:38.922722 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:38.922867 (+ 145us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:38.922875 (+ 8us) write_op.cc:435] Acquired schema lock
0618 13:59:38.922877 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:38.923312 (+ 435us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:38.923327 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:38.923332 (+ 5us) tablet.cc:685] Acquiring locks for 17 operations
0618 13:59:38.923619 (+ 287us) tablet.cc:701] Row locks acquired
0618 13:59:38.923622 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:38.923665 (+ 43us) write_op.cc:270] Start()
0618 13:59:38.923702 (+ 37us) write_op.cc:276] Timestamp: P: 1781791178923635 usec, L: 0
0618 13:59:38.923707 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:38.923922 (+ 215us) log.cc:844] Serialized 6563 byte log entry
0618 13:59:39.008023 (+ 84101us) op_driver.cc:464] REPLICATION: finished
0618 13:59:39.022817 (+ 14794us) write_op.cc:301] APPLY: starting
0618 13:59:39.022843 (+ 26us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:39.022976 (+ 133us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:39.023024 (+ 48us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:39.023444 (+ 420us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:39.023507 (+ 63us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0618 13:59:39.023519 (+ 12us) write_op.cc:312] APPLY: finished
0618 13:59:39.023671 (+ 152us) log.cc:844] Serialized 119 byte log entry
0618 13:59:39.023994 (+ 323us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:39.024044 (+ 50us) write_op.cc:454] Released schema lock
0618 13:59:39.024058 (+ 14us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":106665,"reported_tablets":193,"child_traces":[["op",{"apply.queue_time_us":14690,"apply.run_cpu_time_us":1121,"apply.run_wall_time_us":1348,"num_ops":17,"prepare.queue_time_us":15301,"prepare.run_cpu_time_us":1555,"prepare.run_wall_time_us":36609,"raft.queue_time_us":82,"raft.run_cpu_time_us":587,"raft.run_wall_time_us":588,"replication_time_us":84267,"thread_start_us":167733,"threads_started":3,"wal-append.queue_time_us":27419}]]}
W20260618 13:59:40.239682 29025 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.1:48495 (request call id 149) took 1355 ms (client timeout 1000 ms). Trace:
W20260618 13:59:40.239862 29025 rpcz_store.cc:269] 0618 13:59:38.883868 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.239592 (+1355724us) service_pool.cc:211] Skipping call since client already timed out
0618 13:59:40.239634 (+ 42us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260618 13:59:40.240198 29025 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.2:52853 (request call id 122) took 1314 ms (client timeout 1000 ms). Trace:
W20260618 13:59:40.240367 29025 rpcz_store.cc:269] 0618 13:59:38.926120 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.240106 (+1313986us) service_pool.cc:211] Skipping call since client already timed out
0618 13:59:40.240143 (+ 37us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260618 13:59:40.185997 29746 ts_tablet_manager.cc:1929] T 04dac931e62a448d99633878eb70b92f P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260618 13:59:40.234243 29018 rpcz_store.cc:269] 0618 13:59:36.712315 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:37.486494 (+774179us) service_pool.cc:224] Handling call
0618 13:59:40.179703 (+2693209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63218,"reported_tablets":191}
I20260618 13:59:40.240622 29746 log.cc:1199] T 04dac931e62a448d99633878eb70b92f P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/04dac931e62a448d99633878eb70b92f
I20260618 13:59:40.241127 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "316110bd2d024c7599fa2933d87e559e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.241909 29746 ts_tablet_manager.cc:1950] T 04dac931e62a448d99633878eb70b92f P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:40.242761 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 153) took 5243 ms. Trace:
W20260618 13:59:40.235155 29023 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.1:48495 (request call id 146) took 4632 ms (client timeout 1000 ms). Trace:
I20260618 13:59:40.242854 29746 rpcz_store.cc:276] 0618 13:59:34.999692 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.052933 (+ 53241us) service_pool.cc:224] Handling call
0618 13:59:40.155661 (+5102728us) ts_tablet_manager.cc:1151] Deleting tablet 04dac931e62a448d99633878eb70b92f
0618 13:59:40.155672 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.161216 (+ 5544us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.185937 (+ 24721us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.242744 (+ 56807us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5102563}
I20260618 13:59:40.243194 29746 tablet_replica.cc:333] T 015494374645469785f1f978eaf0e9a5 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:40.243410 29746 raft_consensus.cc:2243] T 015494374645469785f1f978eaf0e9a5 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:40.243564 29746 raft_consensus.cc:2272] T 015494374645469785f1f978eaf0e9a5 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.244079 29746 ts_tablet_manager.cc:1916] T 015494374645469785f1f978eaf0e9a5 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:40.245559 28799 tablet_bootstrap.cc:492] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:40.245707 28799 ts_tablet_manager.cc:1403] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.258s user 0.017s sys 0.072s
I20260618 13:59:40.246697 28799 raft_consensus.cc:359] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.246904 28799 raft_consensus.cc:385] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.246968 28799 raft_consensus.cc:740] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.247202 28799 consensus_queue.cc:260] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.247680 28484 leader_election.cc:304] T 316110bd2d024c7599fa2933d87e559e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.247736 28799 ts_tablet_manager.cc:1434] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:40.247913 30165 raft_consensus.cc:2749] T 316110bd2d024c7599fa2933d87e559e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:40.248013 28799 tablet_bootstrap.cc:492] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:40.249699 28799 tablet_bootstrap.cc:654] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.250928 28797 tablet_bootstrap.cc:492] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:40.251057 28797 ts_tablet_manager.cc:1403] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.042s user 0.003s sys 0.001s
I20260618 13:59:40.251791 28797 raft_consensus.cc:359] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.251961 28797 raft_consensus.cc:385] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.252024 28797 raft_consensus.cc:740] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.252219 28797 consensus_queue.cc:260] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.256839 30152 raft_consensus.cc:493] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.243335 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 04dac931e62a448d99633878eb70b92f (table default.Table_f37d21a9a59249d6a0b87c3788ff4873 [id=22eb9037fc66472793e9fd3f03fd7218]) successfully deleted
I20260618 13:59:40.257133 30152 raft_consensus.cc:515] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.258782 29566 raft_consensus.cc:2272] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:40.242908 29023 rpcz_store.cc:269] 0618 13:59:35.603132 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.663833 (+ 60701us) service_pool.cc:224] Handling call
0618 13:59:40.235111 (+4571278us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:37.852248 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:37.852578 (+ 330us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:37.852583 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:37.852584 (+ 1us) tablet.cc:662] Decoding operations
0618 13:59:37.854485 (+ 1901us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:37.854498 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:37.854508 (+ 10us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:37.854599 (+ 91us) tablet.cc:701] Row locks acquired
0618 13:59:37.854601 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:37.854633 (+ 32us) write_op.cc:270] Start()
0618 13:59:37.854659 (+ 26us) write_op.cc:276] Timestamp: P: 1781791177854607 usec, L: 0
0618 13:59:37.854663 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:37.864447 (+ 9784us) log.cc:844] Serialized 356 byte log entry
0618 13:59:37.864933 (+ 486us) spinlock_profiling.cc:243] Waited 9.4 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fcbac75ef22 00007fcbadb40305 00007fcbadb3ee29 00007fcbadb47250 00007fcbadb4705d 0000000000aa681c 00007fcba07299ff 00007fcba072cd1b 00007fcba072cb98 0000000000aa681c
0618 13:59:37.869239 (+ 4306us) op_driver.cc:464] REPLICATION: finished
0618 13:59:37.869337 (+ 98us) write_op.cc:301] APPLY: starting
0618 13:59:37.869352 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:37.869391 (+ 39us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:37.869394 (+ 3us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:37.869442 (+ 48us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:37.869471 (+ 29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:37.869478 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:37.869544 (+ 66us) log.cc:844] Serialized 23 byte log entry
0618 13:59:37.869597 (+ 53us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:37.869614 (+ 17us) write_op.cc:454] Released schema lock
0618 13:59:37.869620 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:37.869684 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:37.869688 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:39.566572 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:39.566721 (+ 149us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:39.566730 (+ 9us) write_op.cc:435] Acquired schema lock
0618 13:59:39.566734 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:39.566870 (+ 136us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:39.566885 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:39.566892 (+ 7us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:39.566970 (+ 78us) tablet.cc:701] Row locks acquired
0618 13:59:39.566974 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:39.568191 (+ 1217us) write_op.cc:270] Start()
0618 13:59:39.568235 (+ 44us) write_op.cc:276] Timestamp: P: 1781791179566989 usec, L: 0
0618 13:59:39.568240 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:39.568443 (+ 203us) log.cc:844] Serialized 356 byte log entry
0618 13:59:39.741846 (+173403us) op_driver.cc:464] REPLICATION: finished
0618 13:59:39.764777 (+ 22931us) write_op.cc:301] APPLY: starting
0618 13:59:39.764796 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:39.764844 (+ 48us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:39.764871 (+ 27us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:39.764939 (+ 68us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:39.764981 (+ 42us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:39.764989 (+ 8us) write_op.cc:312] APPLY: finished
0618 13:59:39.765089 (+ 100us) log.cc:844] Serialized 23 byte log entry
0618 13:59:39.765176 (+ 87us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:39.765198 (+ 22us) write_op.cc:454] Released schema lock
0618 13:59:39.765207 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":82655,"reported_tablets":186,"child_traces":[["op",{"apply.queue_time_us":51,"apply.run_cpu_time_us":339,"apply.run_wall_time_us":340,"num_ops":1,"prepare.queue_time_us":227445,"prepare.run_cpu_time_us":997,"prepare.run_wall_time_us":13020,"raft.queue_time_us":424,"raft.run_cpu_time_us":527,"raft.run_wall_time_us":527,"replication_time_us":14550,"spinlock_wait_cycles":21630208}],["op",{"apply.queue_time_us":22871,"apply.run_cpu_time_us":526,"apply.run_wall_time_us":1390,"num_ops":1,"prepare.queue_time_us":30103,"prepare.run_cpu_time_us":832,"prepare.run_wall_time_us":3950,"replication_time_us":173574,"thread_start_us":40648,"threads_started":1}]]}
I20260618 13:59:40.259696 30152 leader_election.cc:290] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:40.260196 29023 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_5f54daf83a3347cc885616848fee0f43" } new_table_name: "default.Table_21242eba3cb64c7c91082e296439cfe2" modify_external_catalogs: true
I20260618 13:59:40.259881 30152 raft_consensus.cc:493] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.260960 30152 raft_consensus.cc:515] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:40.261615 30152 leader_election.cc:290] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:40.269634 28798 tablet_bootstrap.cc:492] T bf5e04eb43ed4cf493c0f6f08e22baeb P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:40.269812 28798 ts_tablet_manager.cc:1403] T bf5e04eb43ed4cf493c0f6f08e22baeb P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.075s user 0.002s sys 0.003s
I20260618 13:59:40.270785 28798 raft_consensus.cc:359] T bf5e04eb43ed4cf493c0f6f08e22baeb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.271001 28798 raft_consensus.cc:385] T bf5e04eb43ed4cf493c0f6f08e22baeb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.271072 28798 raft_consensus.cc:740] T bf5e04eb43ed4cf493c0f6f08e22baeb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.273504 29566 ts_tablet_manager.cc:1916] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:40.275460 28797 ts_tablet_manager.cc:1434] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.024s user 0.001s sys 0.001s
I20260618 13:59:40.275959 28797 tablet_bootstrap.cc:492] T eebebb79db644535b54a48e4dee89d59 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:40.277387 28797 tablet_bootstrap.cc:654] T eebebb79db644535b54a48e4dee89d59 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.279841 29566 ts_tablet_manager.cc:1929] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:40.279949 29566 log.cc:1199] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/591465e2b57247a2b224cc8b09225811
I20260618 13:59:40.280265 29566 ts_tablet_manager.cc:1950] T 591465e2b57247a2b224cc8b09225811 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:40.281010 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 101) took 8143 ms. Trace:
I20260618 13:59:40.281106 29566 rpcz_store.cc:276] 0618 13:59:32.137327 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.137369 (+ 42us) service_pool.cc:224] Handling call
0618 13:59:40.070035 (+7932666us) ts_tablet_manager.cc:1151] Deleting tablet 591465e2b57247a2b224cc8b09225811
0618 13:59:40.070046 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.276652 (+206606us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.279797 (+ 3145us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.280985 (+ 1188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14098,"tablet-delete.queue_time_us":7932419}
I20260618 13:59:40.281997 29746 ts_tablet_manager.cc:1929] T 015494374645469785f1f978eaf0e9a5 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:40.282174 29746 log.cc:1199] T 015494374645469785f1f978eaf0e9a5 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/015494374645469785f1f978eaf0e9a5
I20260618 13:59:40.282734 29746 ts_tablet_manager.cc:1950] T 015494374645469785f1f978eaf0e9a5 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:40.283679 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 154) took 5283 ms. Trace:
I20260618 13:59:40.283841 29746 rpcz_store.cc:276] 0618 13:59:35.002752 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.053114 (+ 50362us) service_pool.cc:224] Handling call
0618 13:59:40.243141 (+5190027us) ts_tablet_manager.cc:1151] Deleting tablet 015494374645469785f1f978eaf0e9a5
0618 13:59:40.243158 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.269125 (+ 25967us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.281945 (+ 12820us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.283654 (+ 1709us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5189898}
I20260618 13:59:40.284206 29566 tablet_replica.cc:333] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:40.284536 29566 raft_consensus.cc:2243] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:40.284576 29746 tablet_replica.cc:333] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:40.284909 29746 raft_consensus.cc:2243] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:40.285336 29746 raft_consensus.cc:2272] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.286072 29746 ts_tablet_manager.cc:1916] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:40.287366 29566 raft_consensus.cc:2272] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.290598 29566 ts_tablet_manager.cc:1916] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:40.291262 28799 tablet_bootstrap.cc:492] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
W20260618 13:59:40.291318 28983 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.100.60:42083 (error 108) [suppressed 12 similar messages]
I20260618 13:59:40.291420 28799 ts_tablet_manager.cc:1403] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.043s user 0.001s sys 0.005s
I20260618 13:59:40.292303 28799 raft_consensus.cc:359] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.292564 28799 raft_consensus.cc:385] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.292631 28799 raft_consensus.cc:740] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
W20260618 13:59:40.249310 29019 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.1:48495 (request call id 145) took 5747 ms (client timeout 1000 ms). Trace:
I20260618 13:59:40.292830 28799 consensus_queue.cc:260] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.293133 28798 consensus_queue.cc:260] T bf5e04eb43ed4cf493c0f6f08e22baeb P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.293659 28798 ts_tablet_manager.cc:1434] T bf5e04eb43ed4cf493c0f6f08e22baeb P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.024s user 0.000s sys 0.002s
W20260618 13:59:40.292953 29019 rpcz_store.cc:269] 0618 13:59:34.502286 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.512554 (+ 10268us) service_pool.cc:224] Handling call
0618 13:59:40.249252 (+5736698us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:36.666760 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:36.666903 (+ 143us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:36.666910 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:36.666912 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:36.667072 (+ 160us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:36.667086 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:36.667090 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:36.667223 (+ 133us) tablet.cc:701] Row locks acquired
0618 13:59:36.667226 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:36.667354 (+ 128us) write_op.cc:270] Start()
0618 13:59:36.667454 (+ 100us) write_op.cc:276] Timestamp: P: 1781791176667209 usec, L: 0
0618 13:59:36.667461 (+ 7us) op_driver.cc:348] REPLICATION: starting
0618 13:59:36.667724 (+ 263us) log.cc:844] Serialized 356 byte log entry
0618 13:59:36.716107 (+ 48383us) op_driver.cc:464] REPLICATION: finished
0618 13:59:36.718781 (+ 2674us) write_op.cc:301] APPLY: starting
0618 13:59:36.718802 (+ 21us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:36.718873 (+ 71us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:36.718914 (+ 41us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:36.718992 (+ 78us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:36.719037 (+ 45us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:36.719048 (+ 11us) write_op.cc:312] APPLY: finished
0618 13:59:36.719163 (+ 115us) log.cc:844] Serialized 23 byte log entry
0618 13:59:36.719488 (+ 325us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.719516 (+ 28us) write_op.cc:454] Released schema lock
0618 13:59:36.719527 (+ 11us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0618 13:59:37.586209 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:37.586807 (+ 598us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:37.586817 (+ 10us) write_op.cc:435] Acquired schema lock
0618 13:59:37.586820 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:37.587018 (+ 198us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:37.587036 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:37.587042 (+ 6us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:37.587134 (+ 92us) tablet.cc:701] Row locks acquired
0618 13:59:37.587138 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:37.587186 (+ 48us) write_op.cc:270] Start()
0618 13:59:37.587224 (+ 38us) write_op.cc:276] Timestamp: P: 1781791177587116 usec, L: 0
0618 13:59:37.587227 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:37.587864 (+ 637us) log.cc:844] Serialized 356 byte log entry
0618 13:59:37.604025 (+ 16161us) op_driver.cc:464] REPLICATION: finished
0618 13:59:37.655411 (+ 51386us) write_op.cc:301] APPLY: starting
0618 13:59:37.655453 (+ 42us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:37.655507 (+ 54us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:37.655544 (+ 37us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:37.655623 (+ 79us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:37.655670 (+ 47us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:37.655681 (+ 11us) write_op.cc:312] APPLY: finished
0618 13:59:37.655800 (+ 119us) log.cc:844] Serialized 23 byte log entry
0618 13:59:37.655883 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:37.655904 (+ 21us) write_op.cc:454] Released schema lock
0618 13:59:37.655911 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":152319,"reported_tablets":191,"child_traces":[["op",{"apply.queue_time_us":2589,"apply.run_cpu_time_us":586,"apply.run_wall_time_us":901,"num_ops":1,"prepare.queue_time_us":33400,"prepare.run_cpu_time_us":976,"prepare.run_wall_time_us":2169,"raft.queue_time_us":119,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":309,"replication_time_us":48579,"thread_start_us":54903,"threads_started":2}],["op",{"apply.queue_time_us":51284,"apply.run_cpu_time_us":587,"apply.run_wall_time_us":588,"num_ops":1,"prepare.queue_time_us":109375,"prepare.run_cpu_time_us":1004,"prepare.run_wall_time_us":3192,"raft.queue_time_us":74,"raft.run_cpu_time_us":519,"raft.run_wall_time_us":520,"replication_time_us":16765,"thread_start_us":182042,"threads_started":2}]]}
I20260618 13:59:40.294029 29019 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_c35470a550534fa2ad072f932e981df5" } modify_external_catalogs: true
W20260618 13:59:40.248364 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 9c17e76ae9b44207998a43e3576908c4 because the tablet was not found. No further retry: Not found: Tablet not found: 9c17e76ae9b44207998a43e3576908c4
W20260618 13:59:40.294361 28988 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 45993.7us
W20260618 13:59:40.294633 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 3a0039a463ec4a9c82a21a3a69b565fc because the tablet was not found. No further retry: Not found: Tablet not found: 3a0039a463ec4a9c82a21a3a69b565fc
W20260618 13:59:40.294831 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet b0285c75c632457383e81805ab036aaf because the tablet was not found. No further retry: Not found: Tablet not found: b0285c75c632457383e81805ab036aaf
W20260618 13:59:40.294996 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet d667ea5f029f46d89fcb3be03069b97b because the tablet was not found. No further retry: Not found: Tablet not found: d667ea5f029f46d89fcb3be03069b97b
W20260618 13:59:40.295209 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet aa8cf927bed9437086efd8e0f21ba26a because the tablet was not found. No further retry: Not found: Tablet not found: aa8cf927bed9437086efd8e0f21ba26a
W20260618 13:59:40.295408 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet f45556a59c424ca2bc30956566374c89 because the tablet was not found. No further retry: Not found: Tablet not found: f45556a59c424ca2bc30956566374c89
I20260618 13:59:40.302536 28799 ts_tablet_manager.cc:1434] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260618 13:59:40.302860 28799 tablet_bootstrap.cc:492] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:40.304730 28799 tablet_bootstrap.cc:654] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.307999 30498 tablet.cc:1724] T 7a9bd8577f4f4a9593a3ffeab582b02e P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:40.308604 30165 raft_consensus.cc:493] T bf5e04eb43ed4cf493c0f6f08e22baeb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.308746 30165 raft_consensus.cc:515] T bf5e04eb43ed4cf493c0f6f08e22baeb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.309283 30165 leader_election.cc:290] T bf5e04eb43ed4cf493c0f6f08e22baeb P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:40.310004 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5e04eb43ed4cf493c0f6f08e22baeb" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:40.311854 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5e04eb43ed4cf493c0f6f08e22baeb" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.312394 28797 tablet_bootstrap.cc:492] T eebebb79db644535b54a48e4dee89d59 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:40.312537 28797 ts_tablet_manager.cc:1403] T eebebb79db644535b54a48e4dee89d59 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.001s
I20260618 13:59:40.313396 28797 raft_consensus.cc:359] T eebebb79db644535b54a48e4dee89d59 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.313591 28797 raft_consensus.cc:385] T eebebb79db644535b54a48e4dee89d59 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.313666 28797 raft_consensus.cc:740] T eebebb79db644535b54a48e4dee89d59 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.313900 28797 consensus_queue.cc:260] T eebebb79db644535b54a48e4dee89d59 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.314497 28797 ts_tablet_manager.cc:1434] T eebebb79db644535b54a48e4dee89d59 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:40.314800 28797 tablet_bootstrap.cc:492] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:40.316838 28797 tablet_bootstrap.cc:654] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.318308 28484 leader_election.cc:304] T bf5e04eb43ed4cf493c0f6f08e22baeb P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.318622 30165 raft_consensus.cc:2749] T bf5e04eb43ed4cf493c0f6f08e22baeb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:40.247239 29007 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.2:52853 (request call id 118) took 5857 ms (client timeout 1000 ms). Trace:
W20260618 13:59:40.319455 29007 rpcz_store.cc:269] 0618 13:59:34.390030 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.429020 (+ 38990us) service_pool.cc:224] Handling call
0618 13:59:40.247204 (+5818184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":185359,"reported_tablets":217}
I20260618 13:59:40.323791 29007 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_6024ed5a738d4f91b734fa235807f43f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:40.324128 28798 tablet_bootstrap.cc:492] T 650ed7ee8b134bc4b9f95028c54f99be P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:40.325824 28798 tablet_bootstrap.cc:654] T 650ed7ee8b134bc4b9f95028c54f99be P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.337474 28797 tablet_bootstrap.cc:492] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:40.337714 28797 ts_tablet_manager.cc:1403] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.000s
W20260618 13:59:40.338450 30501 negotiation.cc:336] Failed RPC negotiation. Trace:
0618 13:59:40.302762 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.27.100.60:42083 (local address 127.27.100.1:36777)
0618 13:59:40.337356 (+ 34594us) negotiation.cc:107] Waiting for socket to connect
0618 13:59:40.337937 (+ 581us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":34241,"thread_start_us":33750,"threads_started":1}
W20260618 13:59:40.340051 30502 negotiation.cc:336] Failed RPC negotiation. Trace:
0618 13:59:40.302942 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.27.100.60:42083 (local address 127.27.100.2:37163)
0618 13:59:40.305324 (+ 2382us) negotiation.cc:107] Waiting for socket to connect
0618 13:59:40.305346 (+ 22us) client_negotiation.cc:175] Beginning negotiation
0618 13:59:40.305482 (+ 136us) client_negotiation.cc:253] Sending NEGOTIATE NegotiatePB request
0618 13:59:40.339762 (+ 34280us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":2275,"thread_start_us":2188,"threads_started":1}
I20260618 13:59:40.339285 28797 raft_consensus.cc:359] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.349090 28797 raft_consensus.cc:385] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.349226 28797 raft_consensus.cc:740] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.332890 29746 ts_tablet_manager.cc:1929] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260618 13:59:40.350313 28983 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.60:42083, user_credentials={real_user=slave}} blocked reactor thread for 59031.6us
I20260618 13:59:40.352384 29746 log.cc:1199] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/ac58e939730c4af681c2f08bcfbb6602
I20260618 13:59:40.349586 28797 consensus_queue.cc:260] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.281690 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 591465e2b57247a2b224cc8b09225811 (table default.Table_0a85d79af65f48a3803bfd1248593a11 [id=2f4d62be772a421ab777fd23be58b719]) successfully deleted
W20260618 13:59:40.355324 28989 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.100.3:35503, user_credentials={real_user=slave}} blocked reactor thread for 72821.6us
W20260618 13:59:40.355358 29007 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6024ed5a738d4f91b734fa235807f43f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:40.357374 28797 ts_tablet_manager.cc:1434] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.020s user 0.003s sys 0.000s
I20260618 13:59:40.357496 29746 ts_tablet_manager.cc:1950] T ac58e939730c4af681c2f08bcfbb6602 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:40.359200 28797 tablet_bootstrap.cc:492] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:40.361955 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 015494374645469785f1f978eaf0e9a5 (table default.Table_7699960fdee24242bf51bcfcbd6f68c2 [id=3b9ab803536344d3a59ff77b2b58ae31]) successfully deleted
W20260618 13:59:40.362247 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet b6dd422c05324c989e29b20ae2afabad because the tablet was not found. No further retry: Not found: Tablet not found: b6dd422c05324c989e29b20ae2afabad
I20260618 13:59:40.362186 29566 ts_tablet_manager.cc:1929] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260618 13:59:40.362675 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet d7a97176f72349cda221baaea61d18ad because the tablet was not found. No further retry: Not found: Tablet not found: d7a97176f72349cda221baaea61d18ad
W20260618 13:59:40.363094 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet d4667fe48f0f4eff9c6c8baabdab58d4 because the tablet was not found. No further retry: Not found: Tablet not found: d4667fe48f0f4eff9c6c8baabdab58d4
W20260618 13:59:40.363297 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 8fd9fa9675234db2abff694a517c0ea8 because the tablet was not found. No further retry: Not found: Tablet not found: 8fd9fa9675234db2abff694a517c0ea8
W20260618 13:59:40.363509 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 98b42d23832545ae81a7cdff19e4689f because the tablet was not found. No further retry: Not found: Tablet not found: 98b42d23832545ae81a7cdff19e4689f
W20260618 13:59:40.363693 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 2f5a334d18ba4c06bb358fa86d8af80d because the tablet was not found. No further retry: Not found: Tablet not found: 2f5a334d18ba4c06bb358fa86d8af80d
W20260618 13:59:40.363889 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet b51d0851efb34eaa9e9e4be6b292ef62 because the tablet was not found. No further retry: Not found: Tablet not found: b51d0851efb34eaa9e9e4be6b292ef62
W20260618 13:59:40.364079 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet fe750fc5d998448caeab0a6c808dfba7 because the tablet was not found. No further retry: Not found: Tablet not found: fe750fc5d998448caeab0a6c808dfba7
I20260618 13:59:40.363010 29566 log.cc:1199] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/864f2602840c40fc8606b3784e671cba
I20260618 13:59:40.365711 30506 tablet.cc:1724] T 7a9bd8577f4f4a9593a3ffeab582b02e P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:40.366065 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 155) took 5363 ms. Trace:
W20260618 13:59:40.367043 28983 consensus_peers.cc:597] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 -> Peer 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083): Couldn't send request to peer 7637285570574a6dbcef1de6f84ecac0. Status: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:40.367290 29566 ts_tablet_manager.cc:1950] T 864f2602840c40fc8606b3784e671cba P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:40.367470 29746 rpcz_store.cc:276] 0618 13:59:35.002907 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.053257 (+ 50350us) service_pool.cc:224] Handling call
0618 13:59:40.284517 (+5231260us) ts_tablet_manager.cc:1151] Deleting tablet ac58e939730c4af681c2f08bcfbb6602
0618 13:59:40.284531 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.289306 (+ 4775us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.332594 (+ 43288us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.365882 (+ 33288us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5231136}
I20260618 13:59:40.371313 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet ac58e939730c4af681c2f08bcfbb6602 (table default.Table_ceb33d60d86e4fd898ae67c78da94a55 [id=3251ae257e0f49b99352dcc1ef004588]) successfully deleted
I20260618 13:59:40.372609 29746 tablet_replica.cc:333] T c557afc3abfa4ef188bf248e89f49d87 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:40.373420 29746 raft_consensus.cc:2243] T c557afc3abfa4ef188bf248e89f49d87 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:40.374020 29746 raft_consensus.cc:2272] T c557afc3abfa4ef188bf248e89f49d87 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.375837 28799 tablet_bootstrap.cc:492] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:40.376006 28799 ts_tablet_manager.cc:1403] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.073s user 0.012s sys 0.006s
I20260618 13:59:40.376928 28799 raft_consensus.cc:359] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.377125 28799 raft_consensus.cc:385] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.377182 28799 raft_consensus.cc:740] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.377400 28799 consensus_queue.cc:260] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.377998 30514 tablet.cc:1724] T 2df9e9e811ec4c2ea68467b1294bb62e P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:40.378948 30165 raft_consensus.cc:493] T 657e55fa53c447babfe91b1582806e6b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.379112 30165 raft_consensus.cc:515] T 657e55fa53c447babfe91b1582806e6b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:40.379138 28799 ts_tablet_manager.cc:1434] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:40.379693 30165 leader_election.cc:290] T 657e55fa53c447babfe91b1582806e6b P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:40.379725 28799 tablet_bootstrap.cc:492] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:40.379948 28797 tablet_bootstrap.cc:654] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.380096 29746 ts_tablet_manager.cc:1916] T c557afc3abfa4ef188bf248e89f49d87 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:40.380508 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "657e55fa53c447babfe91b1582806e6b" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.380645 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "657e55fa53c447babfe91b1582806e6b" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:40.381063 28799 tablet_bootstrap.cc:654] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.381217 28484 leader_election.cc:304] T 657e55fa53c447babfe91b1582806e6b P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.381666 30165 raft_consensus.cc:2749] T 657e55fa53c447babfe91b1582806e6b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:40.384985 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 102) took 8246 ms. Trace:
I20260618 13:59:40.385357 29566 rpcz_store.cc:276] 0618 13:59:32.138382 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.138453 (+ 71us) service_pool.cc:224] Handling call
0618 13:59:40.284091 (+8145638us) ts_tablet_manager.cc:1151] Deleting tablet 864f2602840c40fc8606b3784e671cba
0618 13:59:40.284131 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.346133 (+ 62002us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.361839 (+ 15706us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.384744 (+ 22905us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2,"tablet-delete.queue_time_us":8145437}
I20260618 13:59:40.386727 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 864f2602840c40fc8606b3784e671cba (table default.Table_fe8f6240599e467a92887643cc9d210b [id=3ac9a6b120d44ac2bae74d85d7bc6930]) successfully deleted
I20260618 13:59:40.387543 29566 tablet_replica.cc:333] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:40.389969 29566 raft_consensus.cc:2243] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:40.392509 29019 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_c35470a550534fa2ad072f932e981df5" } modify_external_catalogs: true
I20260618 13:59:40.393486 29566 raft_consensus.cc:2272] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.397583 29746 ts_tablet_manager.cc:1929] T c557afc3abfa4ef188bf248e89f49d87 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:40.407550 28799 tablet_bootstrap.cc:492] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:40.407711 28799 ts_tablet_manager.cc:1403] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.001s
I20260618 13:59:40.407990 28548 raft_consensus.cc:3060] T 316110bd2d024c7599fa2933d87e559e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:40.408691 30516 tablet.cc:1724] T 2df9e9e811ec4c2ea68467b1294bb62e P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:40.412315 28799 raft_consensus.cc:359] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.412511 28799 raft_consensus.cc:385] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.412569 28799 raft_consensus.cc:740] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.412781 28799 consensus_queue.cc:260] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.412973 28798 tablet_bootstrap.cc:492] T 650ed7ee8b134bc4b9f95028c54f99be P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:40.414734 30151 raft_consensus.cc:493] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.414880 30151 raft_consensus.cc:515] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.415402 28798 ts_tablet_manager.cc:1403] T 650ed7ee8b134bc4b9f95028c54f99be P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.091s user 0.003s sys 0.001s
I20260618 13:59:40.415450 30151 leader_election.cc:290] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:40.416397 29746 log.cc:1199] T c557afc3abfa4ef188bf248e89f49d87 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/c557afc3abfa4ef188bf248e89f49d87
I20260618 13:59:40.416857 28799 ts_tablet_manager.cc:1434] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260618 13:59:40.417313 29746 ts_tablet_manager.cc:1950] T c557afc3abfa4ef188bf248e89f49d87 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:40.418239 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 156) took 5415 ms. Trace:
I20260618 13:59:40.418671 28799 tablet_bootstrap.cc:492] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:40.419483 29746 rpcz_store.cc:276] 0618 13:59:35.003149 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.053390 (+ 50241us) service_pool.cc:224] Handling call
0618 13:59:40.372217 (+5318827us) ts_tablet_manager.cc:1151] Deleting tablet c557afc3abfa4ef188bf248e89f49d87
0618 13:59:40.372412 (+ 195us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.394827 (+ 22415us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.397530 (+ 2703us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.418221 (+ 20691us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5318705}
I20260618 13:59:40.420534 28799 tablet_bootstrap.cc:654] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.416412 28798 raft_consensus.cc:359] T 650ed7ee8b134bc4b9f95028c54f99be P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.421404 28798 raft_consensus.cc:385] T 650ed7ee8b134bc4b9f95028c54f99be P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.421361 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet c557afc3abfa4ef188bf248e89f49d87 (table default.Table_fdff0afd87584a53a884c9b76926d6d0 [id=7cd5262564144438b00bde60a6cee1e6]) successfully deleted
I20260618 13:59:40.421552 28798 raft_consensus.cc:740] T 650ed7ee8b134bc4b9f95028c54f99be P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.421895 28798 consensus_queue.cc:260] T 650ed7ee8b134bc4b9f95028c54f99be P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.422335 29746 tablet_replica.cc:333] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:40.422825 29746 raft_consensus.cc:2243] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:40.424371 29746 raft_consensus.cc:2272] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.425051 29746 ts_tablet_manager.cc:1916] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:40.424068 28798 ts_tablet_manager.cc:1434] T 650ed7ee8b134bc4b9f95028c54f99be P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260618 13:59:40.432049 28798 tablet_bootstrap.cc:492] T ad045effb32948b5b8e9be5bb7996577 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:40.436360 30514 tablet.cc:1724] T 7a9bd8577f4f4a9593a3ffeab582b02e P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:40.438962 28798 tablet_bootstrap.cc:654] T ad045effb32948b5b8e9be5bb7996577 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.440651 29746 ts_tablet_manager.cc:1929] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:40.441246 29746 log.cc:1199] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/7572d17724d249739be1a1cb9468f909
I20260618 13:59:40.441736 29746 ts_tablet_manager.cc:1950] T 7572d17724d249739be1a1cb9468f909 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:40.443075 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 157) took 5439 ms. Trace:
I20260618 13:59:40.444051 29746 rpcz_store.cc:276] 0618 13:59:35.003249 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.053504 (+ 50255us) service_pool.cc:224] Handling call
0618 13:59:40.422278 (+5368774us) ts_tablet_manager.cc:1151] Deleting tablet 7572d17724d249739be1a1cb9468f909
0618 13:59:40.422290 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.428442 (+ 6152us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.440601 (+ 12159us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.443056 (+ 2455us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5368658}
I20260618 13:59:40.445175 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 7572d17724d249739be1a1cb9468f909 (table default.Table_b583e9f1f2464ebe9df9b4663bb0a4f8 [id=67643f46e18742859d79b0608aeaad23]) successfully deleted
I20260618 13:59:40.445739 29746 tablet_replica.cc:333] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:40.446133 29746 raft_consensus.cc:2243] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:40.446643 29746 raft_consensus.cc:2272] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.447487 29746 ts_tablet_manager.cc:1916] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:40.447860 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b968a8d4014341ca9c5964263721ac79" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:40.449306 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 55ae29c3b7ed44cfa659130745037aa6 because the tablet was not found. No further retry: Not found: Tablet not found: 55ae29c3b7ed44cfa659130745037aa6
I20260618 13:59:40.449548 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bef52c0fc354d6cb5388700641af9f8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:40.449683 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet d1baf86497d442dbbbddbc3498bebbee because the tablet was not found. No further retry: Not found: Tablet not found: d1baf86497d442dbbbddbc3498bebbee
W20260618 13:59:40.450021 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 7aa9f65d8a5c46fdb745ee17af946dce because the tablet was not found. No further retry: Not found: Tablet not found: 7aa9f65d8a5c46fdb745ee17af946dce
W20260618 13:59:40.450378 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet bce7bf74b6d24d0bb43808b8a62cf1b7 because the tablet was not found. No further retry: Not found: Tablet not found: bce7bf74b6d24d0bb43808b8a62cf1b7
I20260618 13:59:40.450757 28798 tablet_bootstrap.cc:492] T ad045effb32948b5b8e9be5bb7996577 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
W20260618 13:59:40.450749 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 9a0691de253b4b008e5d995fd6b876c2 because the tablet was not found. No further retry: Not found: Tablet not found: 9a0691de253b4b008e5d995fd6b876c2
I20260618 13:59:40.450984 28798 ts_tablet_manager.cc:1403] T ad045effb32948b5b8e9be5bb7996577 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.003s
I20260618 13:59:40.451354 29007 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 109) took 5802 ms. Trace:
W20260618 13:59:40.452003 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e because the tablet was not found. No further retry: Not found: Tablet not found: 4d6b01b897134d5bbd6a7d93b82ddb5e
I20260618 13:59:40.451936 28798 raft_consensus.cc:359] T ad045effb32948b5b8e9be5bb7996577 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.452111 28798 raft_consensus.cc:385] T ad045effb32948b5b8e9be5bb7996577 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.452257 28798 raft_consensus.cc:740] T ad045effb32948b5b8e9be5bb7996577 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
W20260618 13:59:40.452414 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet dd2385e6752d49429422fc854539e9c6 because the tablet was not found. No further retry: Not found: Tablet not found: dd2385e6752d49429422fc854539e9c6
I20260618 13:59:40.451556 29007 rpcz_store.cc:276] 0618 13:59:34.648463 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.323387 (+5674924us) service_pool.cc:224] Handling call
0618 13:59:40.392368 (+ 68981us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:40.392955 (+ 587us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:40.428974 (+ 36019us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:40.450945 (+ 21971us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:40.451299 (+ 354us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:40.394034 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:40.394128 (+ 94us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:40.394133 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:40.394136 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:40.394207 (+ 71us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:40.394216 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:40.394220 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:40.394282 (+ 62us) tablet.cc:701] Row locks acquired
0618 13:59:40.394284 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:40.394312 (+ 28us) write_op.cc:270] Start()
0618 13:59:40.394337 (+ 25us) write_op.cc:276] Timestamp: P: 1781791180394292 usec, L: 0
0618 13:59:40.394341 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:40.394515 (+ 174us) log.cc:844] Serialized 433 byte log entry
0618 13:59:40.398942 (+ 4427us) op_driver.cc:464] REPLICATION: finished
0618 13:59:40.399745 (+ 803us) write_op.cc:301] APPLY: starting
0618 13:59:40.399756 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:40.399788 (+ 32us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:40.399816 (+ 28us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:40.399942 (+ 126us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:40.399968 (+ 26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:40.399975 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:40.400034 (+ 59us) log.cc:844] Serialized 29 byte log entry
0618 13:59:40.400073 (+ 39us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:40.400088 (+ 15us) write_op.cc:454] Released schema lock
0618 13:59:40.400093 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":767,"apply.run_cpu_time_us":397,"apply.run_wall_time_us":396,"num_ops":2,"prepare.queue_time_us":566,"prepare.run_cpu_time_us":10471,"prepare.run_wall_time_us":51159,"raft.queue_time_us":930,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":7,"replication_time_us":4573,"thread_start_us":50537,"threads_started":2}]]}
I20260618 13:59:40.452641 28798 consensus_queue.cc:260] T ad045effb32948b5b8e9be5bb7996577 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:40.452724 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet d502d67826ff4d7a8b0f515e5d8b6d77 because the tablet was not found. No further retry: Not found: Tablet not found: d502d67826ff4d7a8b0f515e5d8b6d77
W20260618 13:59:40.453239 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 751cd9bd961b4f56a588ed02a05f0212 because the tablet was not found. No further retry: Not found: Tablet not found: 751cd9bd961b4f56a588ed02a05f0212
W20260618 13:59:40.453454 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet c557afc3abfa4ef188bf248e89f49d87 because the tablet was not found. No further retry: Not found: Tablet not found: c557afc3abfa4ef188bf248e89f49d87
W20260618 13:59:40.453648 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet cd06bb7cffc34412b420d70435fce373 because the tablet was not found. No further retry: Not found: Tablet not found: cd06bb7cffc34412b420d70435fce373
W20260618 13:59:40.453890 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 5e25027f423144b088dad955c7282921 because the tablet was not found. No further retry: Not found: Tablet not found: 5e25027f423144b088dad955c7282921
I20260618 13:59:40.454222 28798 ts_tablet_manager.cc:1434] T ad045effb32948b5b8e9be5bb7996577 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260618 13:59:40.454268 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet e95c88e4b0a046a3ad69ee49af47c7c3 because the tablet was not found. No further retry: Not found: Tablet not found: e95c88e4b0a046a3ad69ee49af47c7c3
W20260618 13:59:40.454613 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 04dac931e62a448d99633878eb70b92f because the tablet was not found. No further retry: Not found: Tablet not found: 04dac931e62a448d99633878eb70b92f
W20260618 13:59:40.454965 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 0fae928833e44d2ba860d6184d4c0142 because the tablet was not found. No further retry: Not found: Tablet not found: 0fae928833e44d2ba860d6184d4c0142
W20260618 13:59:40.455284 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet c7f42997416841a7b2c1ad347d2e9282 because the tablet was not found. No further retry: Not found: Tablet not found: c7f42997416841a7b2c1ad347d2e9282
I20260618 13:59:40.455663 28798 tablet_bootstrap.cc:492] T 538077e459a44ed78c95f2e881b6e0b0 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
W20260618 13:59:40.458959 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet b16f7d3c7837498e996bba1818110642 because the tablet was not found. No further retry: Not found: Tablet not found: b16f7d3c7837498e996bba1818110642
W20260618 13:59:40.459442 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 283523e5b22843ba935eabe6283d3a99 because the tablet was not found. No further retry: Not found: Tablet not found: 283523e5b22843ba935eabe6283d3a99
W20260618 13:59:40.459784 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet aef7c1f111634c98af046e73fa9ffdeb because the tablet was not found. No further retry: Not found: Tablet not found: aef7c1f111634c98af046e73fa9ffdeb
W20260618 13:59:40.460132 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 46bfe8cc2078414d84e65e22ed4ef8a2 because the tablet was not found. No further retry: Not found: Tablet not found: 46bfe8cc2078414d84e65e22ed4ef8a2
W20260618 13:59:40.460445 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 2f3d2f236fd44ab8af0524ea282cfa94 because the tablet was not found. No further retry: Not found: Tablet not found: 2f3d2f236fd44ab8af0524ea282cfa94
W20260618 13:59:40.460738 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 962bf304a285412fb2a529db5b994e9e because the tablet was not found. No further retry: Not found: Tablet not found: 962bf304a285412fb2a529db5b994e9e
W20260618 13:59:40.461094 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 5c75278db2234b11bf95f1cf5cc76edd because the tablet was not found. No further retry: Not found: Tablet not found: 5c75278db2234b11bf95f1cf5cc76edd
W20260618 13:59:40.461406 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 7d4144b1b5ea4e968672616bc22bead8 because the tablet was not found. No further retry: Not found: Tablet not found: 7d4144b1b5ea4e968672616bc22bead8
W20260618 13:59:40.461695 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet f89c9954d48e407c96c294c26644e25c because the tablet was not found. No further retry: Not found: Tablet not found: f89c9954d48e407c96c294c26644e25c
W20260618 13:59:40.461993 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 092a9aba9c6e48f5be9cf5d55ba93678 because the tablet was not found. No further retry: Not found: Tablet not found: 092a9aba9c6e48f5be9cf5d55ba93678
W20260618 13:59:40.462471 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 41107eb1d9164a9c98060003b4027412 because the tablet was not found. No further retry: Not found: Tablet not found: 41107eb1d9164a9c98060003b4027412
W20260618 13:59:40.462786 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet bc0adedecee4405e86b0a37b9e62e352 because the tablet was not found. No further retry: Not found: Tablet not found: bc0adedecee4405e86b0a37b9e62e352
W20260618 13:59:40.463124 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 827cc0987bd541759e334fb620f04ed5 because the tablet was not found. No further retry: Not found: Tablet not found: 827cc0987bd541759e334fb620f04ed5
W20260618 13:59:40.463434 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet e3d8616846c649b6a969b971db0d03db because the tablet was not found. No further retry: Not found: Tablet not found: e3d8616846c649b6a969b971db0d03db
W20260618 13:59:40.463747 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 591465e2b57247a2b224cc8b09225811 because the tablet was not found. No further retry: Not found: Tablet not found: 591465e2b57247a2b224cc8b09225811
I20260618 13:59:40.464020 28798 tablet_bootstrap.cc:654] T 538077e459a44ed78c95f2e881b6e0b0 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:40.464097 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet e6de1293572f4e75af3417f281b9a142 because the tablet was not found. No further retry: Not found: Tablet not found: e6de1293572f4e75af3417f281b9a142
W20260618 13:59:40.465790 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 75824a5d2f7e47838b60e6a2b785c6b2 because the tablet was not found. No further retry: Not found: Tablet not found: 75824a5d2f7e47838b60e6a2b785c6b2
I20260618 13:59:40.466167 30151 raft_consensus.cc:493] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.466298 30151 raft_consensus.cc:515] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.401739 28797 tablet_bootstrap.cc:492] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:40.466593 28797 ts_tablet_manager.cc:1403] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.108s user 0.003s sys 0.004s
I20260618 13:59:40.466874 30151 leader_election.cc:290] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:40.466996 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 02b5cc08ca0c4e719e0414b345211ba5 because the tablet was not found. No further retry: Not found: Tablet not found: 02b5cc08ca0c4e719e0414b345211ba5
W20260618 13:59:40.467236 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet cea4bc13eabc4dd98b4f96723498014c because the tablet was not found. No further retry: Not found: Tablet not found: cea4bc13eabc4dd98b4f96723498014c
I20260618 13:59:40.467458 28797 raft_consensus.cc:359] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.467607 28797 raft_consensus.cc:385] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.467658 28797 raft_consensus.cc:740] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
W20260618 13:59:40.467814 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 6eea4d59935d40aa97dc10ec7469aa56 because the tablet was not found. No further retry: Not found: Tablet not found: 6eea4d59935d40aa97dc10ec7469aa56
I20260618 13:59:40.467829 28797 consensus_queue.cc:260] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:40.468025 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 0463fa2e30554abcbfc8f37b702e81d4 because the tablet was not found. No further retry: Not found: Tablet not found: 0463fa2e30554abcbfc8f37b702e81d4
I20260618 13:59:40.468101 29644 consensus_queue.cc:1048] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:40.468230 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet be59f988f0e34b9480f859cf88b9c223 because the tablet was not found. No further retry: Not found: Tablet not found: be59f988f0e34b9480f859cf88b9c223
I20260618 13:59:40.468328 28797 ts_tablet_manager.cc:1434] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260618 13:59:40.468422 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 6c4a31a4f202473d97b17b5b5dfdb615 because the tablet was not found. No further retry: Not found: Tablet not found: 6c4a31a4f202473d97b17b5b5dfdb615
I20260618 13:59:40.468622 28797 tablet_bootstrap.cc:492] T 8af2bec820b842b4bf6b57482007bd6a P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:40.468612 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 98446f40214c4ac984fcfe98310005c5 because the tablet was not found. No further retry: Not found: Tablet not found: 98446f40214c4ac984fcfe98310005c5
W20260618 13:59:40.468834 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 015494374645469785f1f978eaf0e9a5 because the tablet was not found. No further retry: Not found: Tablet not found: 015494374645469785f1f978eaf0e9a5
W20260618 13:59:40.469035 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 35d8dd2c388e43a0be380aa953ec2c82 because the tablet was not found. No further retry: Not found: Tablet not found: 35d8dd2c388e43a0be380aa953ec2c82
W20260618 13:59:40.469273 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet c3d523419ff244f085d6eac1a3231973 because the tablet was not found. No further retry: Not found: Tablet not found: c3d523419ff244f085d6eac1a3231973
W20260618 13:59:40.469498 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet f3ba9186b3ff47d5a2b158fef619686e because the tablet was not found. No further retry: Not found: Tablet not found: f3ba9186b3ff47d5a2b158fef619686e
W20260618 13:59:40.469691 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 7572d17724d249739be1a1cb9468f909 because the tablet was not found. No further retry: Not found: Tablet not found: 7572d17724d249739be1a1cb9468f909
W20260618 13:59:40.469870 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet ab521028ac1a4d0da013112d76bb169f because the tablet was not found. No further retry: Not found: Tablet not found: ab521028ac1a4d0da013112d76bb169f
W20260618 13:59:40.470055 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 18d8b609b96a4e81b57f69bff522eeea because the tablet was not found. No further retry: Not found: Tablet not found: 18d8b609b96a4e81b57f69bff522eeea
I20260618 13:59:40.470135 28797 tablet_bootstrap.cc:654] T 8af2bec820b842b4bf6b57482007bd6a P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:40.470212 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet ca2ee746c5024cfc9b31c470f9616e7d because the tablet was not found. No further retry: Not found: Tablet not found: ca2ee746c5024cfc9b31c470f9616e7d
W20260618 13:59:40.470405 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet c01c8463055e4526b3c75190066f30b6 because the tablet was not found. No further retry: Not found: Tablet not found: c01c8463055e4526b3c75190066f30b6
W20260618 13:59:40.470610 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 97a9f949999446f09359629fe483bf67 because the tablet was not found. No further retry: Not found: Tablet not found: 97a9f949999446f09359629fe483bf67
W20260618 13:59:40.470865 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet e5f7eaa7d218440ca2c35340c5c87814 because the tablet was not found. No further retry: Not found: Tablet not found: e5f7eaa7d218440ca2c35340c5c87814
I20260618 13:59:40.408326 30513 tablet.cc:1724] T 2df9e9e811ec4c2ea68467b1294bb62e P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:40.471421 28548 raft_consensus.cc:3060] T 650ed7ee8b134bc4b9f95028c54f99be P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:40.414111 28618 consensus_peers.cc:597] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:40.471837 28618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 57771us
I20260618 13:59:40.447229 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bef52c0fc354d6cb5388700641af9f8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.449020 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b968a8d4014341ca9c5964263721ac79" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.474295 28681 raft_consensus.cc:2468] T b968a8d4014341ca9c5964263721ac79 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:40.476598 29644 consensus_queue.cc:1048] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:40.478199 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2a6fabad00642528913e735929e295e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:40.479059 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2a6fabad00642528913e735929e295e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.479650 30151 raft_consensus.cc:493] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.479772 30151 raft_consensus.cc:515] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.480031 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b44c3b0c152442a78edc4736da2cb15c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:40.480321 30151 leader_election.cc:290] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:40.481940 30514 tablet.cc:1724] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:40.481235 29566 ts_tablet_manager.cc:1916] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:40.483549 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b44c3b0c152442a78edc4736da2cb15c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.484341 30151 raft_consensus.cc:493] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260618 13:59:40.484398 28346 leader_election.cc:343] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.484467 30151 raft_consensus.cc:515] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.484555 28346 leader_election.cc:304] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:40.484745 30152 raft_consensus.cc:2804] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:40.484826 30152 raft_consensus.cc:493] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:40.484894 30152 raft_consensus.cc:3060] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:40.485028 30151 leader_election.cc:290] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:40.487498 30152 raft_consensus.cc:515] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:40.484763 28346 leader_election.cc:343] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:40.488674 28346 leader_election.cc:343] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.488848 30152 leader_election.cc:290] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:40.489470 30513 tablet.cc:1724] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:40.492405 29007 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_a5e98286eedd40debca5bb4f5e2ece2a" } new_table_name: "default.Table_01e1e5741bb54f78996857f6b79fc5d7" modify_external_catalogs: true
W20260618 13:59:40.498777 28346 leader_election.cc:343] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:40.502741 28346 leader_election.cc:343] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.503576 28346 leader_election.cc:304] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.504081 30152 raft_consensus.cc:2749] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:40.504477 28346 leader_election.cc:343] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.504901 28346 leader_election.cc:304] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.505316 30152 raft_consensus.cc:2749] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:40.505677 28346 leader_election.cc:343] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.506068 28346 leader_election.cc:304] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.506589 28799 tablet_bootstrap.cc:492] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:40.506717 28799 ts_tablet_manager.cc:1403] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.088s user 0.003s sys 0.008s
I20260618 13:59:40.506847 30152 raft_consensus.cc:2749] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:40.508584 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e196dd3c7fe14524a50152643ddbc2ad" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:40.508935 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e196dd3c7fe14524a50152643ddbc2ad" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.509078 28799 raft_consensus.cc:359] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.509245 28799 raft_consensus.cc:385] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.509290 28799 raft_consensus.cc:740] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.509466 28799 consensus_queue.cc:260] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.511845 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b21dcf74fa4264947ca43cc0026e76" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.512786 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b968a8d4014341ca9c5964263721ac79" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:40.513204 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b968a8d4014341ca9c5964263721ac79" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:40.512776 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b21dcf74fa4264947ca43cc0026e76" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:40.513342 28681 raft_consensus.cc:3060] T b968a8d4014341ca9c5964263721ac79 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:40.513600 28797 tablet_bootstrap.cc:492] T 8af2bec820b842b4bf6b57482007bd6a P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:40.513713 28797 ts_tablet_manager.cc:1403] T 8af2bec820b842b4bf6b57482007bd6a P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.045s user 0.004s sys 0.004s
I20260618 13:59:40.514698 28797 raft_consensus.cc:359] T 8af2bec820b842b4bf6b57482007bd6a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.515949 28797 raft_consensus.cc:385] T 8af2bec820b842b4bf6b57482007bd6a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.516141 28797 raft_consensus.cc:740] T 8af2bec820b842b4bf6b57482007bd6a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.516458 28681 raft_consensus.cc:2468] T b968a8d4014341ca9c5964263721ac79 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:40.517176 28799 ts_tablet_manager.cc:1434] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260618 13:59:40.518527 29303 raft_consensus.cc:493] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.518636 29303 raft_consensus.cc:515] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.519179 29303 leader_election.cc:290] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:40.516455 28797 consensus_queue.cc:260] T 8af2bec820b842b4bf6b57482007bd6a P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.520243 28797 ts_tablet_manager.cc:1434] T 8af2bec820b842b4bf6b57482007bd6a P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260618 13:59:40.520536 28797 tablet_bootstrap.cc:492] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:40.522087 28799 tablet_bootstrap.cc:492] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:40.522284 28797 tablet_bootstrap.cc:654] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.523672 28799 tablet_bootstrap.cc:654] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.524029 29566 ts_tablet_manager.cc:1929] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:40.524209 29566 log.cc:1199] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/2f3d2f236fd44ab8af0524ea282cfa94
I20260618 13:59:40.524655 29566 ts_tablet_manager.cc:1950] T 2f3d2f236fd44ab8af0524ea282cfa94 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:40.526129 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b2f34bc17b4a27a91d6cafa9a96c48" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:40.526471 28407 raft_consensus.cc:2468] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:40.526831 28617 leader_election.cc:304] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:40.527042 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b2f34bc17b4a27a91d6cafa9a96c48" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:40.527164 30362 raft_consensus.cc:2804] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:40.527237 30362 raft_consensus.cc:493] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:40.527305 30362 raft_consensus.cc:3060] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:40.530221 30362 raft_consensus.cc:515] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:40.530553 28618 leader_election.cc:343] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.532150 30152 raft_consensus.cc:493] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.532310 30152 raft_consensus.cc:515] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:40.532564 28346 leader_election.cc:343] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:40.532757 28346 leader_election.cc:343] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.532835 30152 leader_election.cc:290] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:40.532909 28346 leader_election.cc:304] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:40.535558 30362 leader_election.cc:290] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:40.536082 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b2f34bc17b4a27a91d6cafa9a96c48" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:40.536649 28618 leader_election.cc:343] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.532294 28799 tablet_bootstrap.cc:492] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:40.536034 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b2f34bc17b4a27a91d6cafa9a96c48" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:40.539860 28797 tablet_bootstrap.cc:492] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:40.539956 28407 raft_consensus.cc:3060] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:40.539986 28799 ts_tablet_manager.cc:1403] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.000s
I20260618 13:59:40.540136 28797 ts_tablet_manager.cc:1403] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20260618 13:59:40.540699 28799 raft_consensus.cc:359] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.540853 28799 raft_consensus.cc:385] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.540905 28799 raft_consensus.cc:740] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.541086 28799 consensus_queue.cc:260] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.541528 28799 ts_tablet_manager.cc:1434] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 13:59:40.540819 28797 raft_consensus.cc:359] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.541795 28799 tablet_bootstrap.cc:492] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:40.541834 28797 raft_consensus.cc:385] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.541966 28797 raft_consensus.cc:740] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.542259 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 103) took 8401 ms. Trace:
I20260618 13:59:40.542609 29566 rpcz_store.cc:276] 0618 13:59:32.140428 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.140480 (+ 52us) service_pool.cc:224] Handling call
0618 13:59:40.387296 (+8246816us) ts_tablet_manager.cc:1151] Deleting tablet 2f3d2f236fd44ab8af0524ea282cfa94
0618 13:59:40.387404 (+ 108us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.490263 (+102859us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.523970 (+ 33707us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.542246 (+ 18276us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8246486}
I20260618 13:59:40.542469 28797 consensus_queue.cc:260] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.543813 28799 tablet_bootstrap.cc:654] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.543850 28407 raft_consensus.cc:2468] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:40.544226 28617 leader_election.cc:304] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:40.544656 30506 tablet.cc:1724] T 9ead386562eb4fd787c7aee776c2194f P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:40.545068 30274 raft_consensus.cc:2804] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:40.545205 30274 raft_consensus.cc:697] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:40.545473 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 2f3d2f236fd44ab8af0524ea282cfa94 (table default.Table_04faa8df2e7c49549ccf29cfca0905fa [id=e8a3305c5ad142e98b34b6bba79c9735]) successfully deleted
I20260618 13:59:40.545455 30274 consensus_queue.cc:237] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.545670 30152 raft_consensus.cc:2804] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:40.545770 30152 raft_consensus.cc:697] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:40.545974 30152 consensus_queue.cc:237] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:40.547323 29566 tablet_replica.cc:333] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:40.547766 29566 raft_consensus.cc:2243] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:40.547984 29566 raft_consensus.cc:2272] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.548496 29566 ts_tablet_manager.cc:1916] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:40.547585 28797 ts_tablet_manager.cc:1434] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20260618 13:59:40.551965 28797 tablet_bootstrap.cc:492] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:40.554204 28797 tablet_bootstrap.cc:654] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:40.557518 28346 leader_election.cc:343] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.557658 28346 leader_election.cc:304] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.557902 30151 raft_consensus.cc:2749] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:40.558055 28346 leader_election.cc:343] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:40.558312 28618 consensus_peers.cc:597] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 015494374645469785f1f978eaf0e9a5. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:40.560006 30513 tablet.cc:1724] T 9024952ab8094fcda1329562bbde8ef5 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:40.560672 29566 ts_tablet_manager.cc:1929] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:40.561910 29566 log.cc:1199] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/c552d1316e3340f4bcfa39618ae309b8
I20260618 13:59:40.562393 29566 ts_tablet_manager.cc:1950] T c552d1316e3340f4bcfa39618ae309b8 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:40.563309 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:40.563822 29024 catalog_manager.cc:5696] T 80e99a85e83c46afa367e59e5ddb7dc7 P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } }
I20260618 13:59:40.564448 29024 catalog_manager.cc:5696] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:40.564841 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
W20260618 13:59:40.565265 28618 consensus_peers.cc:597] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:40.565275 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:40.565444 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_c70d6c5b0dc94b33b4c5772edd2b9dca [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 2 got 0
I20260618 13:59:40.565492 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 104) took 8424 ms. Trace:
I20260618 13:59:40.565816 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:40.566004 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:40.565773 29566 rpcz_store.cc:276] 0618 13:59:32.141298 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.141339 (+ 41us) service_pool.cc:224] Handling call
0618 13:59:40.547277 (+8405938us) ts_tablet_manager.cc:1151] Deleting tablet c552d1316e3340f4bcfa39618ae309b8
0618 13:59:40.547287 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.556498 (+ 9211us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.560632 (+ 4134us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.565480 (+ 4848us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8405790}
I20260618 13:59:40.566190 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:40.566607 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet c552d1316e3340f4bcfa39618ae309b8 (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]) successfully deleted
I20260618 13:59:40.566955 29566 tablet_replica.cc:333] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:40.567358 29566 raft_consensus.cc:2243] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:40.567878 29566 raft_consensus.cc:2272] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.568645 29566 ts_tablet_manager.cc:1916] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:40.560287 30514 tablet.cc:1724] T 9024952ab8094fcda1329562bbde8ef5 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260618 13:59:40.570099 28346 leader_election.cc:343] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.570196 28346 leader_election.cc:304] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.570367 30262 raft_consensus.cc:2749] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:40.570938 29746 ts_tablet_manager.cc:1929] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:40.571084 29746 log.cc:1199] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/ab521028ac1a4d0da013112d76bb169f
I20260618 13:59:40.571512 29746 ts_tablet_manager.cc:1950] T ab521028ac1a4d0da013112d76bb169f P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:40.571326 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52006500f117417b8a2c979d4aeb077c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.575891 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52006500f117417b8a2c979d4aeb077c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:40.577311 28797 tablet_bootstrap.cc:492] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:40.577474 28797 ts_tablet_manager.cc:1403] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.003s
I20260618 13:59:40.578191 28798 tablet_bootstrap.cc:492] T 538077e459a44ed78c95f2e881b6e0b0 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:40.578169 28797 raft_consensus.cc:359] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.578389 28798 ts_tablet_manager.cc:1403] T 538077e459a44ed78c95f2e881b6e0b0 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.123s user 0.006s sys 0.001s
I20260618 13:59:40.578387 28797 raft_consensus.cc:385] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.578470 28797 raft_consensus.cc:740] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.578714 28797 consensus_queue.cc:260] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.579355 28798 raft_consensus.cc:359] T 538077e459a44ed78c95f2e881b6e0b0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:40.580106 28797 ts_tablet_manager.cc:1434] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260618 13:59:40.581212 28797 tablet_bootstrap.cc:492] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:40.584359 28797 tablet_bootstrap.cc:654] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.586535 28799 tablet_bootstrap.cc:492] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:40.586930 28799 ts_tablet_manager.cc:1403] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.045s user 0.003s sys 0.003s
I20260618 13:59:40.587687 28799 raft_consensus.cc:359] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.594583 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
W20260618 13:59:40.595602 28618 consensus_peers.cc:597] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7572d17724d249739be1a1cb9468f909. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:40.594907 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:40.596052 29024 catalog_manager.cc:5696] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:40.598950 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:40.599215 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:40.599509 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:40.599776 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:40.600044 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:40.600445 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:40.600757 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:40.604530 28798 raft_consensus.cc:385] T 538077e459a44ed78c95f2e881b6e0b0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.604646 28798 raft_consensus.cc:740] T 538077e459a44ed78c95f2e881b6e0b0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.604875 30165 raft_consensus.cc:493] T ad045effb32948b5b8e9be5bb7996577 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.605013 30165 raft_consensus.cc:515] T ad045effb32948b5b8e9be5bb7996577 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.605474 29303 consensus_queue.cc:1048] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:40.605669 30165 leader_election.cc:290] T ad045effb32948b5b8e9be5bb7996577 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:40.605952 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad045effb32948b5b8e9be5bb7996577" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.609225 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:40.609553 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 2128dbec33db453ba51070767a15b21e (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]). Expected version 2 got 0
I20260618 13:59:40.610737 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:40.611140 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:40.582084 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 158) took 5578 ms. Trace:
I20260618 13:59:40.611910 29746 rpcz_store.cc:276] 0618 13:59:35.003346 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.053622 (+ 50276us) service_pool.cc:224] Handling call
0618 13:59:40.445692 (+5392070us) ts_tablet_manager.cc:1151] Deleting tablet ab521028ac1a4d0da013112d76bb169f
0618 13:59:40.445701 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.469748 (+ 24047us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.570855 (+101107us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.582029 (+ 11174us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5391953}
I20260618 13:59:40.613538 29746 tablet_replica.cc:333] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
W20260618 13:59:40.588033 28346 leader_election.cc:343] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.606067 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad045effb32948b5b8e9be5bb7996577" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:40.615736 28483 leader_election.cc:304] T ad045effb32948b5b8e9be5bb7996577 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.611905 30152 raft_consensus.cc:493] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.616452 30165 raft_consensus.cc:2749] T ad045effb32948b5b8e9be5bb7996577 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:40.616535 30152 raft_consensus.cc:515] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:40.616827 28346 consensus_peers.cc:597] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:40.616957 28346 leader_election.cc:343] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.617038 28346 leader_election.cc:304] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.617091 30152 leader_election.cc:290] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:40.613662 28799 raft_consensus.cc:385] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.617241 28799 raft_consensus.cc:740] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.617590 28798 consensus_queue.cc:260] T 538077e459a44ed78c95f2e881b6e0b0 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:40.611584 28990 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet ba53ad1988064049a61fcca3a85f8779 (table default.Table_b7f097c1abe1410a800ce5aa25771740 [id=6c30b030a60b47e2be48dfe6e9cfdfdb], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ba53ad1988064049a61fcca3a85f8779 consensus state has no leader
W20260618 13:59:40.617780 28618 consensus_peers.cc:597] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 04dac931e62a448d99633878eb70b92f. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:40.612361 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet ab521028ac1a4d0da013112d76bb169f (table default.Table_3deacb2c9668491a85bbb09ae3f57d66 [id=431a660bfbed46248a603dc7781cf543]) successfully deleted
I20260618 13:59:40.618108 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:40.618378 28799 consensus_queue.cc:260] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.619499 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1663f9d92e54f1caf3c7137412e495c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.619658 28681 raft_consensus.cc:2468] T f1663f9d92e54f1caf3c7137412e495c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:40.618214 29746 raft_consensus.cc:2243] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:40.619462 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1663f9d92e54f1caf3c7137412e495c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:40.622959 29746 raft_consensus.cc:2272] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.623678 29746 ts_tablet_manager.cc:1916] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:40.618731 30262 raft_consensus.cc:2749] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:40.624607 28799 ts_tablet_manager.cc:1434] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.038s user 0.001s sys 0.000s
I20260618 13:59:40.624871 28799 tablet_bootstrap.cc:492] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:40.625720 28540 raft_consensus.cc:3060] T 9024952ab8094fcda1329562bbde8ef5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:40.630319 28346 leader_election.cc:343] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.630558 28346 leader_election.cc:304] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:40.630795 30151 raft_consensus.cc:2804] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:40.630860 30151 raft_consensus.cc:493] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:40.630919 30151 raft_consensus.cc:3060] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:40.631163 29566 ts_tablet_manager.cc:1929] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:40.631281 29566 log.cc:1199] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/75824a5d2f7e47838b60e6a2b785c6b2
I20260618 13:59:40.631693 29566 ts_tablet_manager.cc:1950] T 75824a5d2f7e47838b60e6a2b785c6b2 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:40.633391 30151 raft_consensus.cc:515] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.633983 30151 leader_election.cc:290] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:40.634732 28799 tablet_bootstrap.cc:654] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:40.635926 28618 consensus_peers.cc:597] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:40.641696 29746 ts_tablet_manager.cc:1929] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:40.642105 30274 raft_consensus.cc:493] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.642329 30274 raft_consensus.cc:515] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.643913 30274 leader_election.cc:290] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:40.643954 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15b1b603f024f3295eb07c98991076d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:40.644135 28402 raft_consensus.cc:2468] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:40.644495 28617 leader_election.cc:304] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:40.644838 30274 raft_consensus.cc:2804] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:40.644980 30274 raft_consensus.cc:493] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:40.645114 30274 raft_consensus.cc:3060] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:40.646251 28799 tablet_bootstrap.cc:492] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:40.646857 28799 ts_tablet_manager.cc:1403] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.000s
I20260618 13:59:40.619799 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:40.652302 28799 raft_consensus.cc:359] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.652482 28799 raft_consensus.cc:385] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.652544 28799 raft_consensus.cc:740] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.652731 28799 consensus_queue.cc:260] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.653234 28799 ts_tablet_manager.cc:1434] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260618 13:59:40.653520 28799 tablet_bootstrap.cc:492] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:40.652734 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:40.656134 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a]). Expected version 1 got 0
I20260618 13:59:40.656416 29024 catalog_manager.cc:5696] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } }
I20260618 13:59:40.657538 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:40.661641 28799 tablet_bootstrap.cc:654] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.644205 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15b1b603f024f3295eb07c98991076d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:40.649276 28483 consensus_peers.cc:597] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c552d1316e3340f4bcfa39618ae309b8. This is attempt 6: this message will repeat every 5th retry.
I20260618 13:59:40.663085 30262 raft_consensus.cc:493] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.663203 30262 raft_consensus.cc:515] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.663751 30262 leader_election.cc:290] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:40.664043 30262 raft_consensus.cc:493] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.664151 30262 raft_consensus.cc:515] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.664484 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1663f9d92e54f1caf3c7137412e495c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:40.664631 30262 leader_election.cc:290] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:40.665544 28346 leader_election.cc:343] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.665928 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ed5c1137434fcc80ed0ca56a323be7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:40.666790 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:40.667093 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
W20260618 13:59:40.667336 28618 leader_election.cc:343] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.667436 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b91f053bcc42bb829bb8180e232c8e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.667559 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1663f9d92e54f1caf3c7137412e495c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:40.667665 28681 raft_consensus.cc:3060] T f1663f9d92e54f1caf3c7137412e495c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:40.669273 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:40.669651 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ed5c1137434fcc80ed0ca56a323be7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.670720 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b91f053bcc42bb829bb8180e232c8e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:40.671730 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:40.672148 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:40.672215 30513 tablet.cc:1724] T 316110bd2d024c7599fa2933d87e559e P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:40.672581 28798 ts_tablet_manager.cc:1434] T 538077e459a44ed78c95f2e881b6e0b0 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.094s user 0.006s sys 0.000s
I20260618 13:59:40.672746 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 72c04f55890648f0a8184f3783243f7c (table default.Table_773be314d7e741fcb4fb6eb7d954802e [id=0d83fc68255c458587390c58b12e00d7]). Expected version 2 got 0
I20260618 13:59:40.673015 28798 tablet_bootstrap.cc:492] T fa24afe9db46408cace50f642f879f4b P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:40.673338 30506 tablet.cc:1724] T 316110bd2d024c7599fa2933d87e559e P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:40.674147 29024 catalog_manager.cc:5696] T d2d54a901e9f4d5c89700979ce0618bf P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:40.674810 28798 tablet_bootstrap.cc:654] T fa24afe9db46408cace50f642f879f4b P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:40.679138 28346 leader_election.cc:343] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.679314 30506 tablet.cc:1724] T 9024952ab8094fcda1329562bbde8ef5 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:40.679514 29746 log.cc:1199] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/9c17e76ae9b44207998a43e3576908c4
I20260618 13:59:40.679971 29746 ts_tablet_manager.cc:1950] T 9c17e76ae9b44207998a43e3576908c4 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:40.680231 28681 raft_consensus.cc:2468] T f1663f9d92e54f1caf3c7137412e495c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
W20260618 13:59:40.680748 28618 consensus_peers.cc:597] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:40.683673 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:40.683707 30362 raft_consensus.cc:493] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.683828 30362 raft_consensus.cc:515] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.683938 29024 catalog_manager.cc:5696] T 054a7afb3b644aa49df78302cf8f02c9 P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:40.684391 30362 leader_election.cc:290] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:40.685232 28617 leader_election.cc:343] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: c01c8463055e4526b3c75190066f30b6
I20260618 13:59:40.684477 29024 catalog_manager.cc:5696] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } }
I20260618 13:59:40.686264 29024 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet ba53ad1988064049a61fcca3a85f8779 (table default.Table_b7f097c1abe1410a800ce5aa25771740 [id=6c30b030a60b47e2be48dfe6e9cfdfdb]). Expected version 1 got 0
W20260618 13:59:40.688340 28346 leader_election.cc:343] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.689054 28346 leader_election.cc:304] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:40.689328 30151 consensus_queue.cc:1048] T 340f6bbf18eb412ca3afec237ff7752d P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:40.689700 30151 raft_consensus.cc:2804] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:40.689806 30151 raft_consensus.cc:697] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:40.690032 30151 consensus_queue.cc:237] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.685916 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01c8463055e4526b3c75190066f30b6" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:40.691285 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 159) took 5687 ms. Trace:
I20260618 13:59:40.691390 29746 rpcz_store.cc:276] 0618 13:59:35.003445 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.053750 (+ 50305us) service_pool.cc:224] Handling call
0618 13:59:40.613431 (+5559681us) ts_tablet_manager.cc:1151] Deleting tablet 9c17e76ae9b44207998a43e3576908c4
0618 13:59:40.613442 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.638097 (+ 24655us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.641582 (+ 3485us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.691240 (+ 49658us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4192,"tablet-delete.queue_time_us":5559416}
W20260618 13:59:40.692072 28618 leader_election.cc:343] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.692188 28618 leader_election.cc:304] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:40.692576 29644 raft_consensus.cc:2749] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:40.693949 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 9c17e76ae9b44207998a43e3576908c4 (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]) successfully deleted
I20260618 13:59:40.692701 28798 tablet_bootstrap.cc:492] T fa24afe9db46408cace50f642f879f4b P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:40.694365 28798 ts_tablet_manager.cc:1403] T fa24afe9db46408cace50f642f879f4b P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.000s
I20260618 13:59:40.695427 28798 raft_consensus.cc:359] T fa24afe9db46408cace50f642f879f4b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.696846 28798 raft_consensus.cc:385] T fa24afe9db46408cace50f642f879f4b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.697161 28798 raft_consensus.cc:740] T fa24afe9db46408cace50f642f879f4b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
W20260618 13:59:40.703583 28346 leader_election.cc:343] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.707017 28730 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:40.705821 28799 tablet_bootstrap.cc:492] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:40.715569 28799 ts_tablet_manager.cc:1403] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.062s user 0.002s sys 0.002s
I20260618 13:59:40.717152 28799 raft_consensus.cc:359] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.717460 28799 raft_consensus.cc:385] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.717576 28799 raft_consensus.cc:740] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.717926 28799 consensus_queue.cc:260] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.719919 28799 ts_tablet_manager.cc:1434] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 13:59:40.721297 28799 tablet_bootstrap.cc:492] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:40.723505 28799 tablet_bootstrap.cc:654] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.729552 28346 leader_election.cc:304] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.729924 30152 raft_consensus.cc:2749] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:40.730433 28346 leader_election.cc:343] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.695899 29746 tablet_replica.cc:333] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:40.730630 28346 leader_election.cc:304] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.730903 30152 raft_consensus.cc:2749] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:40.731027 29746 raft_consensus.cc:2243] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:40.731590 29746 raft_consensus.cc:2272] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.732174 28798 consensus_queue.cc:260] T fa24afe9db46408cace50f642f879f4b P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.734561 28798 ts_tablet_manager.cc:1434] T fa24afe9db46408cace50f642f879f4b P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.040s user 0.002s sys 0.000s
I20260618 13:59:40.735546 28798 tablet_bootstrap.cc:492] T f9f7359c4a144ca89fe3e51d1365a425 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:40.738229 28798 tablet_bootstrap.cc:654] T f9f7359c4a144ca89fe3e51d1365a425 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:40.744691 30545 log.cc:927] Time spent T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257: Append to log took a long time: real 0.074s user 0.000s sys 0.000s
I20260618 13:59:40.733121 29746 ts_tablet_manager.cc:1916] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:40.746345 30274 raft_consensus.cc:515] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.747345 30274 leader_election.cc:290] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:40.747700 30498 tablet.cc:1724] T 09cbe2337dd14da1b8e0629ded7e7ad0 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:40.762140 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 106) took 8617 ms. Trace:
I20260618 13:59:40.763819 28799 tablet_bootstrap.cc:492] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:40.764043 28799 ts_tablet_manager.cc:1403] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.043s user 0.002s sys 0.004s
I20260618 13:59:40.764782 28799 raft_consensus.cc:359] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.765033 28799 raft_consensus.cc:385] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.765120 28799 raft_consensus.cc:740] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.765367 28799 consensus_queue.cc:260] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.765882 28799 ts_tablet_manager.cc:1434] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 13:59:40.767129 28799 tablet_bootstrap.cc:492] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:40.768790 28799 tablet_bootstrap.cc:654] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.776207 29566 rpcz_store.cc:276] 0618 13:59:32.144314 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.144359 (+ 45us) service_pool.cc:224] Handling call
0618 13:59:40.566912 (+8422553us) ts_tablet_manager.cc:1151] Deleting tablet 75824a5d2f7e47838b60e6a2b785c6b2
0618 13:59:40.566921 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.627737 (+ 60816us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.631101 (+ 3364us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.762116 (+131015us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8422286}
I20260618 13:59:40.776690 29566 tablet_replica.cc:333] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:40.776929 29566 raft_consensus.cc:2243] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:40.782478 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 75824a5d2f7e47838b60e6a2b785c6b2 (table default.Table_d838306823ff4e22a68efe0a8cb751ff [id=bbfd0a6a47db4b1b8c534405b1068d5e]) successfully deleted
I20260618 13:59:40.770395 29746 ts_tablet_manager.cc:1929] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:40.783805 29746 log.cc:1199] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/3dd0eaa64271489d92aa380df877f257
I20260618 13:59:40.784797 29746 ts_tablet_manager.cc:1950] T 3dd0eaa64271489d92aa380df877f257 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:40.792217 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 160) took 5788 ms. Trace:
I20260618 13:59:40.796339 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:40.796669 29746 rpcz_store.cc:276] 0618 13:59:35.003523 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.054030 (+ 50507us) service_pool.cc:224] Handling call
0618 13:59:40.695828 (+5641798us) ts_tablet_manager.cc:1151] Deleting tablet 3dd0eaa64271489d92aa380df877f257
0618 13:59:40.695841 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.748505 (+ 52664us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.770300 (+ 21795us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.792160 (+ 21860us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5641655}
I20260618 13:59:40.797724 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 161) took 5794 ms. Trace:
I20260618 13:59:40.797878 29746 rpcz_store.cc:276] 0618 13:59:35.003627 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.054185 (+ 50558us) service_pool.cc:224] Handling call
0618 13:59:40.797516 (+5743331us) ts_tablet_manager.cc:1151] Deleting tablet 5e25027f423144b088dad955c7282921
0618 13:59:40.797533 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.797658 (+ 125us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5743088}
I20260618 13:59:40.798395 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 162) took 5788 ms. Trace:
I20260618 13:59:40.798579 29746 rpcz_store.cc:276] 0618 13:59:35.010049 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.054460 (+ 44411us) service_pool.cc:224] Handling call
0618 13:59:40.798281 (+5743821us) ts_tablet_manager.cc:1151] Deleting tablet 98b42d23832545ae81a7cdff19e4689f
0618 13:59:40.798289 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.798345 (+ 56us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5743671}
I20260618 13:59:40.799280 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 163) took 5788 ms. Trace:
I20260618 13:59:40.799435 29746 rpcz_store.cc:276] 0618 13:59:35.010457 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.054622 (+ 44165us) service_pool.cc:224] Handling call
0618 13:59:40.799150 (+5744528us) ts_tablet_manager.cc:1151] Deleting tablet 9a0691de253b4b008e5d995fd6b876c2
0618 13:59:40.799162 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.799227 (+ 65us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5744349}
I20260618 13:59:40.800073 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 164) took 5789 ms. Trace:
I20260618 13:59:40.800235 29746 rpcz_store.cc:276] 0618 13:59:35.010798 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.054815 (+ 44017us) service_pool.cc:224] Handling call
0618 13:59:40.799901 (+5745086us) ts_tablet_manager.cc:1151] Deleting tablet 55ae29c3b7ed44cfa659130745037aa6
0618 13:59:40.799912 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.800017 (+ 105us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5744962}
I20260618 13:59:40.800834 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 165) took 5789 ms. Trace:
I20260618 13:59:40.797330 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 3dd0eaa64271489d92aa380df877f257 (table default.Table_fd6ef45aeec84b48b8fb7a54db4b4f81 [id=ca38f91f52db4c69a0fb3bc6b8f5cadc]) successfully deleted
W20260618 13:59:40.801484 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 5e25027f423144b088dad955c7282921 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5e25027f423144b088dad955c7282921 already in progress: creating tablet
W20260618 13:59:40.801733 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 98b42d23832545ae81a7cdff19e4689f because the tablet was not found. No further retry: Not found: Tablet not found: 98b42d23832545ae81a7cdff19e4689f
W20260618 13:59:40.801919 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 9a0691de253b4b008e5d995fd6b876c2 because the tablet was not found. No further retry: Not found: Tablet not found: 9a0691de253b4b008e5d995fd6b876c2
W20260618 13:59:40.802148 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 55ae29c3b7ed44cfa659130745037aa6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55ae29c3b7ed44cfa659130745037aa6 already in progress: creating tablet
I20260618 13:59:40.802733 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:40.802922 29746 rpcz_store.cc:276] 0618 13:59:35.011092 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.054950 (+ 43858us) service_pool.cc:224] Handling call
0618 13:59:40.800698 (+5745748us) ts_tablet_manager.cc:1151] Deleting tablet 46bfe8cc2078414d84e65e22ed4ef8a2
0618 13:59:40.800709 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.800781 (+ 72us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5745618}
I20260618 13:59:40.803921 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 166) took 5792 ms. Trace:
W20260618 13:59:40.803651 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 46bfe8cc2078414d84e65e22ed4ef8a2 because the tablet was not found. No further retry: Not found: Tablet not found: 46bfe8cc2078414d84e65e22ed4ef8a2
I20260618 13:59:40.804783 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:40.805022 29746 rpcz_store.cc:276] 0618 13:59:35.011331 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.055086 (+ 43755us) service_pool.cc:224] Handling call
0618 13:59:40.803814 (+5748728us) ts_tablet_manager.cc:1151] Deleting tablet d3dbf13544ce408ab97834aec5862390
0618 13:59:40.803822 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.803877 (+ 55us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5748613}
I20260618 13:59:40.805919 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 167) took 5794 ms. Trace:
I20260618 13:59:40.806288 29746 rpcz_store.cc:276] 0618 13:59:35.011676 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.055214 (+ 43538us) service_pool.cc:224] Handling call
0618 13:59:40.805801 (+5750587us) ts_tablet_manager.cc:1151] Deleting tablet da4d4f03911f4ac099ecc2fa1d8af494
0618 13:59:40.805810 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.805867 (+ 57us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5750468}
W20260618 13:59:40.805637 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d3dbf13544ce408ab97834aec5862390 because the tablet was not found. No further retry: Not found: Tablet not found: d3dbf13544ce408ab97834aec5862390
I20260618 13:59:40.807336 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:40.808193 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 133) took 5815 ms. Trace:
I20260618 13:59:40.811690 28797 tablet_bootstrap.cc:492] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:40.811888 28797 ts_tablet_manager.cc:1403] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.231s user 0.008s sys 0.006s
I20260618 13:59:40.813495 29746 rpcz_store.cc:276] 0618 13:59:34.992877 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.993269 (+ 392us) service_pool.cc:224] Handling call
0618 13:59:40.808019 (+5814750us) ts_tablet_manager.cc:1151] Deleting tablet 97a9f949999446f09359629fe483bf67
0618 13:59:40.808029 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.808137 (+ 108us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5752128}
W20260618 13:59:40.814337 28618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 80003.1us
I20260618 13:59:40.814383 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 134) took 5820 ms. Trace:
W20260618 13:59:40.807840 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet da4d4f03911f4ac099ecc2fa1d8af494 because the tablet was not found. No further retry: Not found: Tablet not found: da4d4f03911f4ac099ecc2fa1d8af494
W20260618 13:59:40.814834 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 97a9f949999446f09359629fe483bf67 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 97a9f949999446f09359629fe483bf67 already in progress: creating tablet
I20260618 13:59:40.815014 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15b1b603f024f3295eb07c98991076d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:40.816232 29566 raft_consensus.cc:2272] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.815848 30506 tablet.cc:1724] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:40.817000 29746 rpcz_store.cc:276] 0618 13:59:34.993646 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.993751 (+ 105us) service_pool.cc:224] Handling call
0618 13:59:40.814208 (+5820457us) ts_tablet_manager.cc:1151] Deleting tablet e5f7eaa7d218440ca2c35340c5c87814
0618 13:59:40.814222 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.814326 (+ 104us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5758253}
W20260618 13:59:40.817682 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet e5f7eaa7d218440ca2c35340c5c87814 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5f7eaa7d218440ca2c35340c5c87814 already in progress: creating tablet
I20260618 13:59:40.818065 29746 tablet_replica.cc:333] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:40.815682 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:40.818578 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:40.818886 29746 raft_consensus.cc:2243] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:40.819350 29746 raft_consensus.cc:2272] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.819941 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:40.820760 29746 ts_tablet_manager.cc:1916] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:40.821339 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:40.824703 30513 tablet.cc:1724] T 4bc87770baa34842a25ed97a19354446 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:40.828943 29746 ts_tablet_manager.cc:1929] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:40.822364 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:40.831889 28797 raft_consensus.cc:359] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.832065 28797 raft_consensus.cc:385] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.832116 28797 raft_consensus.cc:740] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.832268 28797 consensus_queue.cc:260] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.832785 28797 ts_tablet_manager.cc:1434] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.021s user 0.001s sys 0.001s
I20260618 13:59:40.833060 28797 tablet_bootstrap.cc:492] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:40.835201 30151 raft_consensus.cc:493] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.835356 30513 tablet.cc:1724] T 1d8b3cfac0f648cd8b1c51102132c676 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:40.835351 30151 raft_consensus.cc:515] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.835986 30151 leader_election.cc:290] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:40.840337 28797 tablet_bootstrap.cc:654] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.843644 29566 ts_tablet_manager.cc:1916] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:40.852010 30454 sys_catalog.cc:455] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } } }
I20260618 13:59:40.852207 30454 sys_catalog.cc:458] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 [sys.catalog]: This master's current role is: LEADER
I20260618 13:59:40.857605 29644 raft_consensus.cc:493] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.857766 29644 raft_consensus.cc:515] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.858384 30262 raft_consensus.cc:493] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.858450 29644 leader_election.cc:290] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:40.858536 30262 raft_consensus.cc:515] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.859040 30262 leader_election.cc:290] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:40.859017 28618 leader_election.cc:343] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.860365 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:40.860569 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:40.861029 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
W20260618 13:59:40.861084 28618 consensus_peers.cc:597] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:40.861253 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:40.861454 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:40.861634 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:40.861881 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 2128dbec33db453ba51070767a15b21e (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]). Expected version 2 got 0
I20260618 13:59:40.862120 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:40.862947 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:40.864035 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet ba53ad1988064049a61fcca3a85f8779 (table default.Table_b7f097c1abe1410a800ce5aa25771740 [id=6c30b030a60b47e2be48dfe6e9cfdfdb]). Expected version 1 got 0
I20260618 13:59:40.864292 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 72c04f55890648f0a8184f3783243f7c (table default.Table_773be314d7e741fcb4fb6eb7d954802e [id=0d83fc68255c458587390c58b12e00d7]). Expected version 2 got 0
I20260618 13:59:40.865314 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:40.865748 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a]). Expected version 1 got 0
I20260618 13:59:40.866245 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:40.866561 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:40.867046 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:40.867525 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:40.867815 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:40.868052 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:40.868268 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:40.868386 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_c70d6c5b0dc94b33b4c5772edd2b9dca [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 2 got 0
I20260618 13:59:40.868538 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:40.868674 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:40.868803 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:40.836680 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebebb79db644535b54a48e4dee89d59" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:40.872958 28346 leader_election.cc:343] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.848273 30552 raft_consensus.cc:493] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.873252 30552 raft_consensus.cc:515] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.874085 30552 leader_election.cc:290] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:40.853351 28543 raft_consensus.cc:3060] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:40.854432 28537 raft_consensus.cc:3060] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:40.854463 28484 consensus_peers.cc:597] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 -> Peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Couldn't send request to peer d6a169b16ced46ca9cfbc631709bb257. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c552d1316e3340f4bcfa39618ae309b8. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:40.854811 28548 raft_consensus.cc:3060] T 657e55fa53c447babfe91b1582806e6b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:40.856560 30516 tablet.cc:1724] T 1d8b3cfac0f648cd8b1c51102132c676 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:40.859681 28545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9f9649587b4c31951204b8bebe7ed8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:40.879446 28346 leader_election.cc:343] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:40.879741 28483 leader_election.cc:343] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: ca2ee746c5024cfc9b31c470f9616e7d
I20260618 13:59:40.880070 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b21dcf74fa4264947ca43cc0026e76" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:40.856685 30506 tablet.cc:1724] T 4bc87770baa34842a25ed97a19354446 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:40.882614 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca2ee746c5024cfc9b31c470f9616e7d" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:40.883230 28618 leader_election.cc:343] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.883553 28484 leader_election.cc:304] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.883831 30550 raft_consensus.cc:2749] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:40.884023 30513 tablet.cc:1724] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:40.885365 30516 tablet.cc:1724] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:40.892191 29746 log.cc:1199] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/283523e5b22843ba935eabe6283d3a99
I20260618 13:59:40.892992 29746 ts_tablet_manager.cc:1950] T 283523e5b22843ba935eabe6283d3a99 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
W20260618 13:59:40.895267 28618 consensus_peers.cc:597] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:40.897083 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 135) took 5902 ms. Trace:
I20260618 13:59:40.897727 29746 rpcz_store.cc:276] 0618 13:59:34.994344 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.994666 (+ 322us) service_pool.cc:224] Handling call
0618 13:59:40.817980 (+5823314us) ts_tablet_manager.cc:1151] Deleting tablet 283523e5b22843ba935eabe6283d3a99
0618 13:59:40.817989 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.826227 (+ 8238us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.828853 (+ 2626us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.897028 (+ 68175us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5761978}
I20260618 13:59:40.898572 30585 raft_consensus.cc:493] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.898777 30585 raft_consensus.cc:515] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.899585 30585 leader_election.cc:290] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:40.899858 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9559c5f2c9e94e858093dd2f0865c865" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:40.900231 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 283523e5b22843ba935eabe6283d3a99 (table default.Table_d16c53f18154437e8d06e88da66faf13 [id=cf2c2a972733485a9ba1d6f1e99937a1]) successfully deleted
W20260618 13:59:40.900364 28618 leader_election.cc:343] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.900913 29746 tablet_replica.cc:333] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:40.901675 29746 raft_consensus.cc:2243] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:40.902536 29746 raft_consensus.cc:2272] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.903559 29746 ts_tablet_manager.cc:1916] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:40.903497 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15b1b603f024f3295eb07c98991076d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:40.903705 28407 raft_consensus.cc:3060] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:40.904441 30514 tablet.cc:1724] T 4bc87770baa34842a25ed97a19354446 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:40.907719 28407 raft_consensus.cc:2468] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:40.918632 30516 tablet.cc:1724] T be00bfa9bc1a4b83b18bfad05ba1495e P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:40.919412 30498 tablet.cc:1724] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:40.926267 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b21dcf74fa4264947ca43cc0026e76" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:40.926463 28403 raft_consensus.cc:2468] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:40.926517 30262 raft_consensus.cc:493] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.926606 30262 raft_consensus.cc:515] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.927001 28797 tablet_bootstrap.cc:492] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:40.927187 30262 leader_election.cc:290] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:40.927559 28797 ts_tablet_manager.cc:1403] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.095s user 0.015s sys 0.011s
I20260618 13:59:40.927784 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2d6894bf5649fc8cfae7e033c432b9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:40.928545 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9559c5f2c9e94e858093dd2f0865c865" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:40.928572 28797 raft_consensus.cc:359] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:40.928723 28797 raft_consensus.cc:385] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.928766 28797 raft_consensus.cc:740] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.928747 28403 raft_consensus.cc:2468] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:40.929237 28346 leader_election.cc:343] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.929270 29400 raft_consensus.cc:493] T eebebb79db644535b54a48e4dee89d59 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.929380 29400 raft_consensus.cc:515] T eebebb79db644535b54a48e4dee89d59 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.929530 28797 consensus_queue.cc:260] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:40.930080 29400 leader_election.cc:290] T eebebb79db644535b54a48e4dee89d59 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:40.931082 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebebb79db644535b54a48e4dee89d59" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:40.931552 28618 leader_election.cc:343] T eebebb79db644535b54a48e4dee89d59 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.931751 28797 ts_tablet_manager.cc:1434] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 13:59:40.932776 28797 tablet_bootstrap.cc:492] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:40.934265 28797 tablet_bootstrap.cc:654] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:40.942005 28346 consensus_peers.cc:597] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:40.942153 28596 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:40.935376 28617 leader_election.cc:304] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:40.935473 30572 consensus_queue.cc:1048] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:40.935496 29400 consensus_queue.cc:1048] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:40.943468 29400 raft_consensus.cc:2804] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:40.943640 29400 raft_consensus.cc:697] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:40.943981 30262 raft_consensus.cc:493] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:40.944120 30262 raft_consensus.cc:515] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.944757 30262 leader_election.cc:290] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:40.945123 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af2bec820b842b4bf6b57482007bd6a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:40.945520 28346 leader_election.cc:343] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.945510 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9f9649587b4c31951204b8bebe7ed8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.945590 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebebb79db644535b54a48e4dee89d59" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.945693 28680 raft_consensus.cc:2468] T eebebb79db644535b54a48e4dee89d59 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:40.945732 29400 consensus_queue.cc:237] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.946545 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2d6894bf5649fc8cfae7e033c432b9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.948536 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af2bec820b842b4bf6b57482007bd6a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:40.948709 28682 raft_consensus.cc:2468] T 8af2bec820b842b4bf6b57482007bd6a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
W20260618 13:59:40.949716 28346 leader_election.cc:343] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.949844 28346 leader_election.cc:304] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.950054 30262 raft_consensus.cc:2749] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:40.935575 28618 consensus_peers.cc:597] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:40.953918 28983 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.60:42083, user_credentials={real_user=slave}} blocked reactor thread for 98144.7us
I20260618 13:59:40.954916 30572 consensus_queue.cc:1048] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:40.955772 28617 leader_election.cc:304] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:40.956485 28346 leader_election.cc:304] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:40.956717 30262 raft_consensus.cc:2804] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:40.956785 30262 raft_consensus.cc:493] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:40.956879 30262 raft_consensus.cc:3060] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:40.957345 28346 leader_election.cc:343] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.957439 28346 leader_election.cc:304] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:40.958611 30152 raft_consensus.cc:2749] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:40.958720 30572 raft_consensus.cc:2804] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:40.958799 30572 raft_consensus.cc:493] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:40.958876 30572 raft_consensus.cc:3060] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:40.958793 30152 consensus_queue.cc:1048] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:40.961319 30572 raft_consensus.cc:515] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.961632 30262 raft_consensus.cc:515] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.961958 30572 leader_election.cc:290] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:40.962266 30262 leader_election.cc:290] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:40.962298 28617 leader_election.cc:304] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:40.965209 29746 ts_tablet_manager.cc:1929] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:40.968317 29746 log.cc:1199] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/ca2ee746c5024cfc9b31c470f9616e7d
I20260618 13:59:40.968715 29746 ts_tablet_manager.cc:1950] T ca2ee746c5024cfc9b31c470f9616e7d P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:40.969071 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 136) took 5973 ms. Trace:
I20260618 13:59:40.969125 28799 tablet_bootstrap.cc:492] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:40.969251 28799 ts_tablet_manager.cc:1403] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.202s user 0.015s sys 0.028s
I20260618 13:59:40.972164 29746 rpcz_store.cc:276] 0618 13:59:34.995536 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.995600 (+ 64us) service_pool.cc:224] Handling call
0618 13:59:40.900815 (+5905215us) ts_tablet_manager.cc:1151] Deleting tablet ca2ee746c5024cfc9b31c470f9616e7d
0618 13:59:40.900827 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.937792 (+ 36965us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.965115 (+ 27323us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.969055 (+ 3940us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5844754}
I20260618 13:59:40.973143 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet ca2ee746c5024cfc9b31c470f9616e7d (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]) successfully deleted
I20260618 13:59:40.973287 28346 leader_election.cc:304] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:40.973824 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebebb79db644535b54a48e4dee89d59" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:40.974192 30603 raft_consensus.cc:2804] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:40.974336 30603 raft_consensus.cc:493] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:40.974628 30603 raft_consensus.cc:3060] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:40.975564 28346 leader_election.cc:343] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:40.968878 30498 tablet.cc:1724] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:40.975801 29746 tablet_replica.cc:333] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:40.976023 29746 raft_consensus.cc:2243] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:40.976203 29746 raft_consensus.cc:2272] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.976720 29746 ts_tablet_manager.cc:1916] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:40.977030 30572 raft_consensus.cc:2804] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:40.977126 30572 raft_consensus.cc:493] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:40.977198 30572 raft_consensus.cc:3060] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:40.978350 29018 catalog_manager.cc:5696] T 7766014ee9814f01ae7ac3e849ca3d50 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:40.980058 30572 raft_consensus.cc:515] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.980566 30572 leader_election.cc:290] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:40.980676 28798 tablet_bootstrap.cc:492] T f9f7359c4a144ca89fe3e51d1365a425 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:40.983371 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebebb79db644535b54a48e4dee89d59" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:40.983575 28403 raft_consensus.cc:2393] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d6a169b16ced46ca9cfbc631709bb257 in current term 1: Already voted for candidate 6fc808af0e884ed8836561665d8e0e7b in this term.
I20260618 13:59:40.983649 28458 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:40.983785 28799 raft_consensus.cc:359] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.984200 28799 raft_consensus.cc:385] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.984076 28798 ts_tablet_manager.cc:1403] T f9f7359c4a144ca89fe3e51d1365a425 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.249s user 0.005s sys 0.022s
I20260618 13:59:40.984809 28799 raft_consensus.cc:740] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.985146 28799 consensus_queue.cc:260] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:40.985756 28798 raft_consensus.cc:359] T f9f7359c4a144ca89fe3e51d1365a425 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:40.986308 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]). Expected version 1 got 0
I20260618 13:59:40.986855 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 1d8b3cfac0f648cd8b1c51102132c676 (table default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3]). Expected version 2 got 0
I20260618 13:59:40.987475 28799 ts_tablet_manager.cc:1434] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260618 13:59:40.987900 28798 raft_consensus.cc:385] T f9f7359c4a144ca89fe3e51d1365a425 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:40.988435 28798 raft_consensus.cc:740] T f9f7359c4a144ca89fe3e51d1365a425 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:40.988770 28799 tablet_bootstrap.cc:492] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:40.989056 28798 consensus_queue.cc:260] T f9f7359c4a144ca89fe3e51d1365a425 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:40.991398 29566 ts_tablet_manager.cc:1929] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:40.993059 28798 ts_tablet_manager.cc:1434] T f9f7359c4a144ca89fe3e51d1365a425 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260618 13:59:40.994553 29566 log.cc:1199] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/02b5cc08ca0c4e719e0414b345211ba5
I20260618 13:59:40.994905 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet ba53ad1988064049a61fcca3a85f8779 (table default.Table_b7f097c1abe1410a800ce5aa25771740 [id=6c30b030a60b47e2be48dfe6e9cfdfdb]). Expected version 1 got 0
I20260618 13:59:40.995229 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:40.996229 28798 tablet_bootstrap.cc:492] T 438663f365e34fab87e34f434920cecf P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:40.996666 29566 ts_tablet_manager.cc:1950] T 02b5cc08ca0c4e719e0414b345211ba5 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:40.998121 28798 tablet_bootstrap.cc:654] T 438663f365e34fab87e34f434920cecf P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:40.995456 29018 catalog_manager.cc:5696] T aa93f065b4d64e7b85e30f0690fae541 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:40.998955 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:40.998984 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 107) took 8853 ms. Trace:
I20260618 13:59:40.999053 29566 rpcz_store.cc:276] 0618 13:59:32.145190 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.145229 (+ 39us) service_pool.cc:224] Handling call
0618 13:59:40.776640 (+8631411us) ts_tablet_manager.cc:1151] Deleting tablet 02b5cc08ca0c4e719e0414b345211ba5
0618 13:59:40.776651 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.849096 (+ 72445us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.991332 (+142236us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.998966 (+ 7634us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8631215}
I20260618 13:59:40.999745 29018 catalog_manager.cc:5696] T 2d1073b726ef4cbeac6c874fc8863e90 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:41.000348 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:41.001032 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:41.002755 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:41.003026 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:41.003299 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:41.003530 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_a5e98286eedd40debca5bb4f5e2ece2a [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 1 got 0
I20260618 13:59:41.004034 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 9024952ab8094fcda1329562bbde8ef5 (table default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f]). Expected version 2 got 0
I20260618 13:59:41.004474 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:41.004973 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 4bc87770baa34842a25ed97a19354446 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]). Expected version 2 got 0
I20260618 13:59:41.005362 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 2128dbec33db453ba51070767a15b21e (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]). Expected version 2 got 0
I20260618 13:59:41.005759 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:41.006695 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:41.009768 30514 tablet.cc:1724] T 09cbe2337dd14da1b8e0629ded7e7ad0 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:41.011186 30603 raft_consensus.cc:515] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.012130 30603 leader_election.cc:290] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.012689 28799 tablet_bootstrap.cc:654] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.013185 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 02b5cc08ca0c4e719e0414b345211ba5 (table default.Table_6f3c7c1d308e4901887fc9b602b38e4f [id=7aa3803c01134af6b073952eb9843282]) successfully deleted
I20260618 13:59:41.013640 29566 tablet_replica.cc:333] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:41.014253 29566 raft_consensus.cc:2243] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:41.015445 29566 raft_consensus.cc:2272] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:40.998461 29746 ts_tablet_manager.cc:1929] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:41.001547 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b21dcf74fa4264947ca43cc0026e76" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:41.009481 30552 raft_consensus.cc:493] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6fc808af0e884ed8836561665d8e0e7b)
I20260618 13:59:41.012161 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af2bec820b842b4bf6b57482007bd6a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:41.014284 30516 tablet.cc:1724] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:41.015781 30552 raft_consensus.cc:515] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:41.015697 29746 log.cc:1199] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/2f3d2f236fd44ab8af0524ea282cfa94
I20260618 13:59:41.014690 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9559c5f2c9e94e858093dd2f0865c865" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:41.016390 29746 ts_tablet_manager.cc:1950] T 2f3d2f236fd44ab8af0524ea282cfa94 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:41.016355 30552 leader_election.cc:290] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:41.016712 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 145) took 6017 ms. Trace:
W20260618 13:59:41.017449 28483 leader_election.cc:343] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 591465e2b57247a2b224cc8b09225811
W20260618 13:59:41.017398 28618 leader_election.cc:343] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.019244 29566 ts_tablet_manager.cc:1916] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.020143 29746 rpcz_store.cc:276] 0618 13:59:34.998812 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.998871 (+ 59us) service_pool.cc:224] Handling call
0618 13:59:40.975759 (+5976888us) ts_tablet_manager.cc:1151] Deleting tablet 2f3d2f236fd44ab8af0524ea282cfa94
0618 13:59:40.975768 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:40.981350 (+ 5582us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:40.998391 (+ 17041us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.016698 (+ 18307us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5919639}
I20260618 13:59:41.020668 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 2f3d2f236fd44ab8af0524ea282cfa94 (table default.Table_04faa8df2e7c49549ccf29cfca0905fa [id=e8a3305c5ad142e98b34b6bba79c9735]) successfully deleted
I20260618 13:59:41.021263 30514 tablet.cc:1724] T 316110bd2d024c7599fa2933d87e559e P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:41.022637 29746 tablet_replica.cc:333] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:41.024025 29746 raft_consensus.cc:2243] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260618 13:59:41.024417 28346 leader_election.cc:343] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.024680 29746 raft_consensus.cc:2272] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.025345 29746 ts_tablet_manager.cc:1916] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.026252 30514 tablet.cc:1724] T 1d8b3cfac0f648cd8b1c51102132c676 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:41.027065 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebebb79db644535b54a48e4dee89d59" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:41.027211 28682 raft_consensus.cc:3060] T eebebb79db644535b54a48e4dee89d59 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.029597 28682 raft_consensus.cc:2468] T eebebb79db644535b54a48e4dee89d59 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
W20260618 13:59:41.033445 28618 leader_election.cc:343] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.033798 28799 tablet_bootstrap.cc:492] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:41.034165 28799 ts_tablet_manager.cc:1403] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.045s user 0.002s sys 0.002s
I20260618 13:59:41.035246 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:41.035885 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_c70d6c5b0dc94b33b4c5772edd2b9dca [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 2 got 0
I20260618 13:59:41.036717 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
W20260618 13:59:41.037518 28484 leader_election.cc:343] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 591465e2b57247a2b224cc8b09225811
I20260618 13:59:41.037672 28484 leader_election.cc:304] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.037956 30552 raft_consensus.cc:2749] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:41.037613 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 72c04f55890648f0a8184f3783243f7c (table default.Table_773be314d7e741fcb4fb6eb7d954802e [id=0d83fc68255c458587390c58b12e00d7]). Expected version 2 got 0
I20260618 13:59:41.035300 28799 raft_consensus.cc:359] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.039047 28799 raft_consensus.cc:385] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.039077 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:41.039628 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 2df9e9e811ec4c2ea68467b1294bb62e (table default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a]). Expected version 1 got 0
I20260618 13:59:41.039789 28799 raft_consensus.cc:740] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.039836 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:41.040226 28797 tablet_bootstrap.cc:492] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:41.040483 28799 consensus_queue.cc:260] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.040563 28797 ts_tablet_manager.cc:1403] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.108s user 0.001s sys 0.005s
I20260618 13:59:41.041383 28797 raft_consensus.cc:359] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.044067 30552 raft_consensus.cc:493] T f9f7359c4a144ca89fe3e51d1365a425 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.044204 30552 raft_consensus.cc:515] T f9f7359c4a144ca89fe3e51d1365a425 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:41.044503 28346 consensus_peers.cc:597] T c38a2156a55e4e5eb91d101e9a61eecd P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:41.044766 30552 leader_election.cc:290] T f9f7359c4a144ca89fe3e51d1365a425 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.045046 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f7359c4a144ca89fe3e51d1365a425" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:41.045945 28799 ts_tablet_manager.cc:1434] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260618 13:59:41.046584 28798 tablet_bootstrap.cc:492] T 438663f365e34fab87e34f434920cecf P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:41.047480 28799 tablet_bootstrap.cc:492] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:41.047597 28797 raft_consensus.cc:385] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.047647 28798 ts_tablet_manager.cc:1403] T 438663f365e34fab87e34f434920cecf P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.051s user 0.003s sys 0.003s
I20260618 13:59:41.047747 28797 raft_consensus.cc:740] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.048035 28797 consensus_queue.cc:260] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.048395 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:41.049278 30552 raft_consensus.cc:493] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6fc808af0e884ed8836561665d8e0e7b)
I20260618 13:59:41.049379 30552 raft_consensus.cc:515] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.049698 28797 ts_tablet_manager.cc:1434] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260618 13:59:41.047924 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f7359c4a144ca89fe3e51d1365a425" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.049893 30552 leader_election.cc:290] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:41.048393 28618 consensus_peers.cc:597] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 283523e5b22843ba935eabe6283d3a99. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:41.051118 28483 leader_election.cc:343] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: cea4bc13eabc4dd98b4f96723498014c
I20260618 13:59:41.052480 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 7a9bd8577f4f4a9593a3ffeab582b02e (table default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a]). Expected version 1 got 0
I20260618 13:59:41.052891 29746 ts_tablet_manager.cc:1929] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:41.053112 28799 tablet_bootstrap.cc:654] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.054183 29746 log.cc:1199] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/e6de1293572f4e75af3417f281b9a142
I20260618 13:59:41.054725 29746 ts_tablet_manager.cc:1950] T e6de1293572f4e75af3417f281b9a142 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:41.054987 28797 tablet_bootstrap.cc:492] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:41.053910 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:41.055704 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 09cbe2337dd14da1b8e0629ded7e7ad0 (table default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6]). Expected version 1 got 0
I20260618 13:59:41.055706 30514 tablet.cc:1724] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:41.049213 28798 raft_consensus.cc:359] T 438663f365e34fab87e34f434920cecf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.056051 28798 raft_consensus.cc:385] T 438663f365e34fab87e34f434920cecf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.056152 28798 raft_consensus.cc:740] T 438663f365e34fab87e34f434920cecf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.056432 28798 consensus_queue.cc:260] T 438663f365e34fab87e34f434920cecf P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.059830 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 148) took 6060 ms. Trace:
I20260618 13:59:41.059924 29746 rpcz_store.cc:276] 0618 13:59:34.999158 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.999217 (+ 59us) service_pool.cc:224] Handling call
0618 13:59:41.022596 (+6023379us) ts_tablet_manager.cc:1151] Deleting tablet e6de1293572f4e75af3417f281b9a142
0618 13:59:41.022605 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.032683 (+ 10078us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.052826 (+ 20143us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.059814 (+ 6988us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5966341}
I20260618 13:59:41.060679 28484 leader_election.cc:304] T f9f7359c4a144ca89fe3e51d1365a425 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.060585 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet e6de1293572f4e75af3417f281b9a142 (table default.Table_f75651a1e55f4271b39388713dcd74aa [id=6705bb69561d435496d424c0d64f9879]) successfully deleted
I20260618 13:59:41.061465 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:41.063047 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:41.063421 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:41.063844 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:41.064225 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:41.064808 29019 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet f2b21dcf74fa4264947ca43cc0026e76
I20260618 13:59:41.060959 30552 raft_consensus.cc:2749] T f9f7359c4a144ca89fe3e51d1365a425 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:41.060771 28798 ts_tablet_manager.cc:1434] T 438663f365e34fab87e34f434920cecf P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260618 13:59:41.065428 29019 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 108) took 6558 ms. Trace:
I20260618 13:59:41.065992 28798 tablet_bootstrap.cc:492] T aa93f065b4d64e7b85e30f0690fae541 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:41.065541 29019 rpcz_store.cc:276] 0618 13:59:34.506630 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.293827 (+5787197us) service_pool.cc:224] Handling call
0618 13:59:40.392733 (+ 98906us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:40.392888 (+ 155us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:40.392992 (+ 104us) catalog_manager.cc:2849] Locking tablets
0618 13:59:40.393161 (+ 169us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:40.525790 (+132629us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:40.525906 (+ 116us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:41.064719 (+538813us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:41.065382 (+ 663us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:40.445271 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:40.445440 (+ 169us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:40.445448 (+ 8us) write_op.cc:435] Acquired schema lock
0618 13:59:40.445453 (+ 5us) tablet.cc:662] Decoding operations
0618 13:59:40.445638 (+ 185us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:40.445659 (+ 21us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:40.445667 (+ 8us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:40.445816 (+ 149us) tablet.cc:701] Row locks acquired
0618 13:59:40.445820 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:40.445878 (+ 58us) write_op.cc:270] Start()
0618 13:59:40.445923 (+ 45us) write_op.cc:276] Timestamp: P: 1781791180445835 usec, L: 0
0618 13:59:40.445929 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:40.497240 (+ 51311us) log.cc:844] Serialized 728 byte log entry
0618 13:59:40.498004 (+ 764us) spinlock_profiling.cc:243] Waited 51.1 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fcbac75ef22 00007fcbadb40305 00007fcbadb3ee29 00007fcbadb47250 00007fcbadb4705d 0000000000aa681c 00007fcba07299ff 00007fcba072cd1b 00007fcba072cb98 0000000000aa681c
0618 13:59:40.523914 (+ 25910us) op_driver.cc:464] REPLICATION: finished
0618 13:59:40.524257 (+ 343us) write_op.cc:301] APPLY: starting
0618 13:59:40.524279 (+ 22us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:40.524335 (+ 56us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:40.524340 (+ 5us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:40.524427 (+ 87us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:40.524482 (+ 55us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:40.524494 (+ 12us) write_op.cc:312] APPLY: finished
0618 13:59:40.524582 (+ 88us) log.cc:844] Serialized 29 byte log entry
0618 13:59:40.524659 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:40.524682 (+ 23us) write_op.cc:454] Released schema lock
0618 13:59:40.524692 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:40.524779 (+ 87us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:40.524785 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":276,"apply.run_cpu_time_us":509,"apply.run_wall_time_us":509,"num_ops":2,"prepare.queue_time_us":51527,"prepare.run_cpu_time_us":1419,"prepare.run_wall_time_us":52824,"raft.queue_time_us":2093,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":285,"replication_time_us":77939,"spinlock_wait_cycles":117487872}]]}
I20260618 13:59:41.066406 29019 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_773be314d7e741fcb4fb6eb7d954802e" } modify_external_catalogs: true
I20260618 13:59:41.066650 29019 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_773be314d7e741fcb4fb6eb7d954802e" } modify_external_catalogs: true
I20260618 13:59:41.067509 28798 tablet_bootstrap.cc:654] T aa93f065b4d64e7b85e30f0690fae541 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.071574 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet f2b21dcf74fa4264947ca43cc0026e76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:40 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:41.071889 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b21dcf74fa4264947ca43cc0026e76" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:41.072108 28402 raft_consensus.cc:3060] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.073246 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet e5f7eaa7d218440ca2c35340c5c87814 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.073431 28512 tablet_service.cc:1558] Processing DeleteTablet for tablet ca2ee746c5024cfc9b31c470f9616e7d with delete_type TABLET_DATA_DELETED (replaced by tablet 60b6216474b74a14af6abf977e312b77) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.073562 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 3dd0eaa64271489d92aa380df877f257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.073709 28512 tablet_service.cc:1558] Processing DeleteTablet for tablet ab521028ac1a4d0da013112d76bb169f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.073834 28525 tablet_service.cc:1558] Processing DeleteTablet for tablet ac58e939730c4af681c2f08bcfbb6602 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.073953 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet f2b21dcf74fa4264947ca43cc0026e76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:40 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.074028 28512 tablet_service.cc:1558] Processing DeleteTablet for tablet 015494374645469785f1f978eaf0e9a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 2e493f647a344cd581515a77230d765f) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.074004 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9559c5f2c9e94e858093dd2f0865c865" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:41.074144 28411 raft_consensus.cc:3060] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.074172 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet 97a9f949999446f09359629fe483bf67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:29 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.074319 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet 283523e5b22843ba935eabe6283d3a99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.073956 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 7572d17724d249739be1a1cb9468f909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:27 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.073987 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e25027f423144b088dad955c7282921 with delete_type TABLET_DATA_DELETED (replaced by tablet a8da96fc621c47d4a444b5a39259f85d) from {username='slave'} at 127.0.0.1:57282
W20260618 13:59:41.075338 29023 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c9ceb3af08fa4618b81cf6f89d7bd0df consensus state has no leader
I20260618 13:59:41.075548 29023 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 107) took 6600 ms. Trace:
I20260618 13:59:41.075670 29023 rpcz_store.cc:276] 0618 13:59:34.475474 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.259995 (+5784521us) service_pool.cc:224] Handling call
0618 13:59:40.260440 (+ 445us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:40.260648 (+ 208us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:40.392283 (+131635us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:40.392400 (+ 117us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:40.434311 (+ 41911us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:41.075500 (+641189us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:40.393157 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:40.393330 (+ 173us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:40.393336 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:40.393338 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:40.393432 (+ 94us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:40.393446 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:40.393450 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:40.393511 (+ 61us) tablet.cc:701] Row locks acquired
0618 13:59:40.393513 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:40.393548 (+ 35us) write_op.cc:270] Start()
0618 13:59:40.393582 (+ 34us) write_op.cc:276] Timestamp: P: 1781791180393525 usec, L: 0
0618 13:59:40.393585 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:40.393806 (+ 221us) log.cc:844] Serialized 405 byte log entry
0618 13:59:40.398843 (+ 5037us) op_driver.cc:464] REPLICATION: finished
0618 13:59:40.399225 (+ 382us) write_op.cc:301] APPLY: starting
0618 13:59:40.399242 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:40.399299 (+ 57us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:40.399336 (+ 37us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:40.399392 (+ 56us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:40.399432 (+ 40us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:40.399439 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:40.399512 (+ 73us) log.cc:844] Serialized 23 byte log entry
0618 13:59:40.399575 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:40.399590 (+ 15us) write_op.cc:454] Released schema lock
0618 13:59:40.399597 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:40.399674 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:40.399679 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":312,"apply.run_cpu_time_us":438,"apply.run_wall_time_us":438,"mutex_wait_us":68,"num_ops":1,"prepare.queue_time_us":56,"prepare.run_cpu_time_us":791,"prepare.run_wall_time_us":854,"raft.queue_time_us":877,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":265,"replication_time_us":5222}]]}
I20260618 13:59:41.076696 28411 raft_consensus.cc:2468] T 9559c5f2c9e94e858093dd2f0865c865 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:41.076866 29023 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_6492eae424c54e20b41a06cd5db24684"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:41.077565 29023 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6492eae424c54e20b41a06cd5db24684 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:41.078060 30514 tablet.cc:1724] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:41.078493 30603 raft_consensus.cc:493] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.078599 30603 raft_consensus.cc:515] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.079478 30603 leader_election.cc:290] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.079634 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c04f55890648f0a8184f3783243f7c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.081908 28402 raft_consensus.cc:2468] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:41.082055 28797 tablet_bootstrap.cc:654] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.060384 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea4bc13eabc4dd98b4f96723498014c" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.085350 28484 leader_election.cc:304] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.085584 30552 raft_consensus.cc:2749] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:41.085609 30618 tablet_replica.cc:333] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:41.085788 30618 raft_consensus.cc:2243] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:41.085978 30618 raft_consensus.cc:2272] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.086544 30618 ts_tablet_manager.cc:1916] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.072424 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet f2b21dcf74fa4264947ca43cc0026e76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:40 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.055572 29566 ts_tablet_manager.cc:1929] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:41.087797 29566 log.cc:1199] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/6eea4d59935d40aa97dc10ec7469aa56
I20260618 13:59:41.088287 29566 ts_tablet_manager.cc:1950] T 6eea4d59935d40aa97dc10ec7469aa56 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:41.089001 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 108) took 8942 ms. Trace:
I20260618 13:59:41.089133 29566 rpcz_store.cc:276] 0618 13:59:32.146559 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.146601 (+ 42us) service_pool.cc:224] Handling call
0618 13:59:41.013577 (+8866976us) ts_tablet_manager.cc:1151] Deleting tablet 6eea4d59935d40aa97dc10ec7469aa56
0618 13:59:41.013591 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.028833 (+ 15242us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.055521 (+ 26688us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.088983 (+ 33462us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8866663}
I20260618 13:59:41.089634 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af2bec820b842b4bf6b57482007bd6a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:41.089804 28680 raft_consensus.cc:3060] T 8af2bec820b842b4bf6b57482007bd6a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.090677 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c04f55890648f0a8184f3783243f7c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.091552 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 6eea4d59935d40aa97dc10ec7469aa56 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]) successfully deleted
I20260618 13:59:41.092725 30572 raft_consensus.cc:493] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.092851 30572 raft_consensus.cc:515] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.093292 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet 04dac931e62a448d99633878eb70b92f with delete_type TABLET_DATA_DELETED (replaced by tablet 15483938a06e459789f87c6bb456bd15) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.093396 30572 leader_election.cc:290] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:41.093627 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet 9c17e76ae9b44207998a43e3576908c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 2128dbec33db453ba51070767a15b21e) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.093788 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 55ae29c3b7ed44cfa659130745037aa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:31 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.093946 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f3d2f236fd44ab8af0524ea282cfa94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.094066 30552 raft_consensus.cc:493] T fa24afe9db46408cace50f642f879f4b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.094162 30552 raft_consensus.cc:515] T fa24afe9db46408cace50f642f879f4b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.094736 30552 leader_election.cc:290] T fa24afe9db46408cace50f642f879f4b P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:41.093547 28680 raft_consensus.cc:2468] T 8af2bec820b842b4bf6b57482007bd6a P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:41.095341 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet c557afc3abfa4ef188bf248e89f49d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.095980 28401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa24afe9db46408cace50f642f879f4b" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:41.097416 28798 tablet_bootstrap.cc:492] T aa93f065b4d64e7b85e30f0690fae541 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:41.097581 28798 ts_tablet_manager.cc:1403] T aa93f065b4d64e7b85e30f0690fae541 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.000s
I20260618 13:59:41.098183 28798 raft_consensus.cc:359] T aa93f065b4d64e7b85e30f0690fae541 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.098407 28798 raft_consensus.cc:385] T aa93f065b4d64e7b85e30f0690fae541 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.100107 28798 raft_consensus.cc:740] T aa93f065b4d64e7b85e30f0690fae541 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.100466 28798 consensus_queue.cc:260] T aa93f065b4d64e7b85e30f0690fae541 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.101253 28798 ts_tablet_manager.cc:1434] T aa93f065b4d64e7b85e30f0690fae541 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260618 13:59:41.103587 29023 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 112) took 5713 ms. Trace:
I20260618 13:59:41.061837 29746 tablet_replica.cc:333] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:41.103834 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet e6de1293572f4e75af3417f281b9a142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.103874 29746 raft_consensus.cc:2243] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:41.104051 29746 raft_consensus.cc:2272] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.104342 28346 leader_election.cc:304] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:41.104569 29566 tablet_replica.cc:333] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:41.104808 29566 raft_consensus.cc:2243] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:41.104997 29566 raft_consensus.cc:2272] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.105546 30151 raft_consensus.cc:2804] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:41.105648 29566 ts_tablet_manager.cc:1916] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.105657 30151 raft_consensus.cc:697] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:41.105875 30151 consensus_queue.cc:237] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:41.108704 28346 leader_election.cc:343] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.108886 28346 leader_election.cc:304] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
W20260618 13:59:41.109180 28346 leader_election.cc:343] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.109258 28346 leader_election.cc:304] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.109530 29746 ts_tablet_manager.cc:1916] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.107970 29023 rpcz_store.cc:276] 0618 13:59:35.389686 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.076593 (+5686907us) service_pool.cc:224] Handling call
0618 13:59:41.077654 (+ 1061us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:41.078202 (+ 548us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:41.103356 (+ 25154us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:41.103462 (+ 106us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:41.103549 (+ 87us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.091028 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.091175 (+ 147us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.091182 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:41.091184 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:41.091296 (+ 112us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.091313 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.091321 (+ 8us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:41.091418 (+ 97us) tablet.cc:701] Row locks acquired
0618 13:59:41.091421 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:41.091457 (+ 36us) write_op.cc:270] Start()
0618 13:59:41.091492 (+ 35us) write_op.cc:276] Timestamp: P: 1781791181091418 usec, L: 0
0618 13:59:41.091496 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.091685 (+ 189us) log.cc:844] Serialized 433 byte log entry
0618 13:59:41.099423 (+ 7738us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.099572 (+ 149us) write_op.cc:301] APPLY: starting
0618 13:59:41.099587 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.099634 (+ 47us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.099665 (+ 31us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.099778 (+ 113us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.099814 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:41.099822 (+ 8us) write_op.cc:312] APPLY: finished
0618 13:59:41.099911 (+ 89us) log.cc:844] Serialized 29 byte log entry
0618 13:59:41.099979 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.099999 (+ 20us) write_op.cc:454] Released schema lock
0618 13:59:41.100005 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:41.100076 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.100080 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":100,"apply.run_cpu_time_us":494,"apply.run_wall_time_us":494,"num_ops":2,"prepare.queue_time_us":323,"prepare.run_cpu_time_us":774,"prepare.run_wall_time_us":775,"raft.queue_time_us":795,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":216,"replication_time_us":7896}]]}
W20260618 13:59:41.114322 28618 consensus_peers.cc:597] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9c17e76ae9b44207998a43e3576908c4. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:41.115734 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa24afe9db46408cace50f642f879f4b" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.116317 30152 raft_consensus.cc:2804] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:41.116389 30513 tablet.cc:1724] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:41.116818 30152 raft_consensus.cc:697] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:41.117007 28798 tablet_bootstrap.cc:492] T 9fa1a27fc9304e18b96cad9f923779f2 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:41.118157 30152 consensus_queue.cc:237] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.119392 30618 ts_tablet_manager.cc:1929] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:41.119560 30618 log.cc:1199] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/f2b21dcf74fa4264947ca43cc0026e76
I20260618 13:59:41.120152 30618 ts_tablet_manager.cc:1950] T f2b21dcf74fa4264947ca43cc0026e76 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
W20260618 13:59:41.123522 29007 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3f5d4296952e48a48756e6f27140da87 consensus state has no leader
I20260618 13:59:41.125097 28798 tablet_bootstrap.cc:654] T 9fa1a27fc9304e18b96cad9f923779f2 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.128170 28548 raft_consensus.cc:3060] T bf5e04eb43ed4cf493c0f6f08e22baeb P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.128398 30603 raft_consensus.cc:2749] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:41.128597 30603 raft_consensus.cc:493] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.128691 30603 raft_consensus.cc:515] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.129204 30603 leader_election.cc:290] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:41.130910 29026 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.2:52853 (request call id 120) took 4447 ms (client timeout 1000 ms). Trace:
I20260618 13:59:41.131269 29007 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 110) took 6444 ms. Trace:
I20260618 13:59:41.138259 28617 leader_election.cc:304] T eebebb79db644535b54a48e4dee89d59 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:41.139071 29303 raft_consensus.cc:2749] T eebebb79db644535b54a48e4dee89d59 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:41.139743 30498 tablet.cc:1724] T 538077e459a44ed78c95f2e881b6e0b0 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:41.139676 29007 rpcz_store.cc:276] 0618 13:59:34.686463 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.453933 (+5767470us) service_pool.cc:224] Handling call
0618 13:59:40.492314 (+ 38381us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb4686956 00007fcbab0e4146 00007fcbab0e3e75 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:40.493021 (+ 707us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:40.493262 (+ 241us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:40.493465 (+ 203us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:40.493527 (+ 62us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:40.525288 (+ 31761us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:41.131236 (+605948us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:40.498125 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:40.498240 (+ 115us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:40.498246 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:40.498249 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:40.498357 (+ 108us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:40.498368 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:40.498373 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:40.498471 (+ 98us) tablet.cc:701] Row locks acquired
0618 13:59:40.498475 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:40.498521 (+ 46us) write_op.cc:270] Start()
0618 13:59:40.498568 (+ 47us) write_op.cc:276] Timestamp: P: 1781791180498485 usec, L: 0
0618 13:59:40.498573 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:40.498768 (+ 195us) log.cc:844] Serialized 405 byte log entry
0618 13:59:40.524007 (+ 25239us) op_driver.cc:464] REPLICATION: finished
0618 13:59:40.524870 (+ 863us) write_op.cc:301] APPLY: starting
0618 13:59:40.524884 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:40.524920 (+ 36us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:40.524954 (+ 34us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:40.524987 (+ 33us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:40.525035 (+ 48us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:40.525046 (+ 11us) write_op.cc:312] APPLY: finished
0618 13:59:40.525110 (+ 64us) log.cc:844] Serialized 23 byte log entry
0618 13:59:40.525161 (+ 51us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:40.525183 (+ 22us) write_op.cc:454] Released schema lock
0618 13:59:40.525191 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:40.525279 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:40.525284 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7539,"spinlock_wait_cycles":32589184,"child_traces":[["op",{"apply.queue_time_us":821,"apply.run_cpu_time_us":398,"apply.run_wall_time_us":397,"num_ops":1,"prepare.queue_time_us":4086,"prepare.run_cpu_time_us":771,"prepare.run_wall_time_us":9623,"raft.queue_time_us":114,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":25409}]]}
I20260618 13:59:41.141945 28617 leader_election.cc:304] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:41.142170 29400 raft_consensus.cc:2804] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:41.142277 29400 raft_consensus.cc:697] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:41.142380 28617 leader_election.cc:304] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:41.142526 29400 consensus_queue.cc:237] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.142632 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f45556a59c424ca2bc30956566374c89" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.144315 30603 raft_consensus.cc:493] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.144554 30603 raft_consensus.cc:515] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.145431 30603 leader_election.cc:290] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.145627 30603 raft_consensus.cc:493] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.145776 30603 raft_consensus.cc:515] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.146482 30603 leader_election.cc:290] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.149760 28798 tablet_bootstrap.cc:492] T 9fa1a27fc9304e18b96cad9f923779f2 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:41.149953 28798 ts_tablet_manager.cc:1403] T 9fa1a27fc9304e18b96cad9f923779f2 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.033s user 0.004s sys 0.000s
W20260618 13:59:41.150048 29011 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.2:52853 (request call id 119) took 5603 ms (client timeout 1000 ms). Trace:
I20260618 13:59:41.150068 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f3d2f236fd44ab8af0524ea282cfa94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:41.150380 29026 rpcz_store.cc:269] 0618 13:59:36.683475 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:37.407111 (+723636us) service_pool.cc:224] Handling call
0618 13:59:41.130868 (+3723757us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":238715,"reported_tablets":220}
I20260618 13:59:41.150880 28798 raft_consensus.cc:359] T 9fa1a27fc9304e18b96cad9f923779f2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.151129 28798 raft_consensus.cc:385] T 9fa1a27fc9304e18b96cad9f923779f2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.151232 28798 raft_consensus.cc:740] T 9fa1a27fc9304e18b96cad9f923779f2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.151449 28798 consensus_queue.cc:260] T 9fa1a27fc9304e18b96cad9f923779f2 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.152540 30516 tablet.cc:1724] T 957670718c8c4b8cb87115cc4a2fa861 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:41.154667 28798 ts_tablet_manager.cc:1434] T 9fa1a27fc9304e18b96cad9f923779f2 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260618 13:59:41.155198 28798 tablet_bootstrap.cc:492] T 7766014ee9814f01ae7ac3e849ca3d50 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:41.156893 28798 tablet_bootstrap.cc:654] T 7766014ee9814f01ae7ac3e849ca3d50 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:41.157807 28618 consensus_peers.cc:597] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab521028ac1a4d0da013112d76bb169f. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:41.162312 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet c552d1316e3340f4bcfa39618ae309b8 with delete_type TABLET_DATA_DELETED (replaced by tablet fe750fc5d998448caeab0a6c808dfba7) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.162698 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 75824a5d2f7e47838b60e6a2b785c6b2 with delete_type TABLET_DATA_DELETED (replaced by tablet d4539f767c054e9e8ebc1f3f56108e7e) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.163033 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 827cc0987bd541759e334fb620f04ed5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.163384 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 864f2602840c40fc8606b3784e671cba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.164296 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 591465e2b57247a2b224cc8b09225811 with delete_type TABLET_DATA_DELETED (replaced by tablet d4667fe48f0f4eff9c6c8baabdab58d4) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.164898 28484 leader_election.cc:304] T fa24afe9db46408cace50f642f879f4b P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.165146 30552 raft_consensus.cc:2749] T fa24afe9db46408cace50f642f879f4b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:41.165158 30577 raft_consensus.cc:2804] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:41.165266 30577 raft_consensus.cc:697] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
W20260618 13:59:41.165514 29011 rpcz_store.cc:269] 0618 13:59:35.537293 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.553068 (+ 15775us) service_pool.cc:224] Handling call
0618 13:59:41.140509 (+5587441us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34720,"reported_tablets":213}
I20260618 13:59:41.165529 30577 consensus_queue.cc:237] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.167356 29026 master_service.cc:960] ReplaceTablet: received request to replace tablet ba2d6894bf5649fc8cfae7e033c432b9 from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:41.170185 30552 raft_consensus.cc:493] T 438663f365e34fab87e34f434920cecf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.170369 30552 raft_consensus.cc:515] T 438663f365e34fab87e34f434920cecf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.171129 30552 leader_election.cc:290] T 438663f365e34fab87e34f434920cecf P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:41.172685 28618 leader_election.cc:343] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.174005 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 02b5cc08ca0c4e719e0414b345211ba5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.174923 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 6eea4d59935d40aa97dc10ec7469aa56 with delete_type TABLET_DATA_DELETED (replaced by tablet 4bc87770baa34842a25ed97a19354446) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.171828 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "438663f365e34fab87e34f434920cecf" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:41.176259 30151 raft_consensus.cc:493] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.177822 30151 raft_consensus.cc:515] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.178663 30151 leader_election.cc:290] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:41.179418 28617 leader_election.cc:343] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: f45556a59c424ca2bc30956566374c89
I20260618 13:59:41.180282 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "438663f365e34fab87e34f434920cecf" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.180290 28617 leader_election.cc:304] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:41.181108 30576 raft_consensus.cc:2749] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:41.189042 30514 tablet.cc:1724] T 538077e459a44ed78c95f2e881b6e0b0 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:41.190325 28048 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu
/tmp/dist-test-taskAN1Nhn/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.100.60:42083
--webserver_interface=127.27.100.60
--webserver_port=38293
--builtin_ntp_servers=127.27.100.20:36797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.100.62:43001,127.27.100.61:45967,127.27.100.60:42083
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260618 13:59:41.200965 28797 tablet_bootstrap.cc:492] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:41.201223 28797 ts_tablet_manager.cc:1403] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.146s user 0.003s sys 0.003s
I20260618 13:59:41.202168 28797 raft_consensus.cc:359] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.203310 28797 raft_consensus.cc:385] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.203632 28797 raft_consensus.cc:740] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.204250 28797 consensus_queue.cc:260] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.205531 30151 raft_consensus.cc:493] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.205703 30151 raft_consensus.cc:515] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.206285 30151 leader_election.cc:290] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:41.207458 28346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 98266.8us
I20260618 13:59:41.208431 28799 tablet_bootstrap.cc:492] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:41.208559 28799 ts_tablet_manager.cc:1403] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.161s user 0.002s sys 0.002s
I20260618 13:59:41.208925 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba18e879631947f88bd266681746ec82" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.209242 28799 raft_consensus.cc:359] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.209405 28799 raft_consensus.cc:385] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.209465 28799 raft_consensus.cc:740] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.209661 28799 consensus_queue.cc:260] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.209802 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e6f3f8431cd49b5bf1306bc7609761d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.210160 28799 ts_tablet_manager.cc:1434] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 13:59:41.210587 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed648dcf882340388155068d7a5a7869" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.210928 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba18e879631947f88bd266681746ec82" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.211388 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed648dcf882340388155068d7a5a7869" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.211470 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e6f3f8431cd49b5bf1306bc7609761d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.211544 28683 raft_consensus.cc:2468] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:41.211789 30572 consensus_queue.cc:1048] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:41.212102 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08132883fb4f4632af2961bead244b18" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.212532 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet f2b21dcf74fa4264947ca43cc0026e76 (table default.Table_c35470a550534fa2ad072f932e981df5 [id=2797c7fefad74c1e82ab31fb9ad4db70]) successfully deleted
I20260618 13:59:41.181809 28484 leader_election.cc:304] T 438663f365e34fab87e34f434920cecf P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.214543 30551 raft_consensus.cc:2749] T 438663f365e34fab87e34f434920cecf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:41.216508 30635 tablet.cc:1724] T be00bfa9bc1a4b83b18bfad05ba1495e P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:41.218762 28797 ts_tablet_manager.cc:1434] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.017s user 0.001s sys 0.001s
I20260618 13:59:41.221477 29746 ts_tablet_manager.cc:1929] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:41.222394 28797 tablet_bootstrap.cc:492] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:41.224238 28797 tablet_bootstrap.cc:654] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.231683 29566 ts_tablet_manager.cc:1929] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:41.231865 29566 log.cc:1199] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/0463fa2e30554abcbfc8f37b702e81d4
I20260618 13:59:41.232275 29566 ts_tablet_manager.cc:1950] T 0463fa2e30554abcbfc8f37b702e81d4 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:41.233031 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 109) took 9084 ms. Trace:
I20260618 13:59:41.233178 29566 rpcz_store.cc:276] 0618 13:59:32.148428 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.148484 (+ 56us) service_pool.cc:224] Handling call
0618 13:59:41.089448 (+8940964us) ts_tablet_manager.cc:1151] Deleting tablet 0463fa2e30554abcbfc8f37b702e81d4
0618 13:59:41.089583 (+ 135us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.108650 (+ 19067us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.231629 (+122979us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.233011 (+ 1382us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8940765}
I20260618 13:59:41.233678 29566 tablet_replica.cc:333] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:41.233975 29566 raft_consensus.cc:2243] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:41.234445 29566 raft_consensus.cc:2272] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.235158 29566 ts_tablet_manager.cc:1916] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.239243 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 0463fa2e30554abcbfc8f37b702e81d4 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]) successfully deleted
I20260618 13:59:41.248454 30633 tablet.cc:1724] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:41.230021 28548 raft_consensus.cc:3060] T fa24afe9db46408cace50f642f879f4b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.243466 28546 raft_consensus.cc:3060] T ad045effb32948b5b8e9be5bb7996577 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.270032 30551 raft_consensus.cc:493] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6fc808af0e884ed8836561665d8e0e7b)
I20260618 13:59:41.270267 30551 raft_consensus.cc:515] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:41.271147 30551 leader_election.cc:290] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:41.272008 29019 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 72c04f55890648f0a8184f3783243f7c
W20260618 13:59:41.273676 28483 leader_election.cc:343] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: be59f988f0e34b9480f859cf88b9c223
I20260618 13:59:41.283273 29026 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet ba2d6894bf5649fc8cfae7e033c432b9
I20260618 13:59:41.284016 29026 catalog_manager.cc:6784] ReplaceTablet: tablet ba2d6894bf5649fc8cfae7e033c432b9 deleted and replaced by tablet a6ba4976bb6e4c2da3a563ccad4eb2b0
I20260618 13:59:41.285041 29026 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 113) took 5883 ms. Trace:
I20260618 13:59:41.285069 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet 72c04f55890648f0a8184f3783243f7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.285166 29026 rpcz_store.cc:276] 0618 13:59:35.401033 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.151096 (+5750063us) service_pool.cc:224] Handling call
0618 13:59:41.284131 (+133035us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.228733 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.228901 (+ 168us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.228910 (+ 9us) write_op.cc:435] Acquired schema lock
0618 13:59:41.228914 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:41.229073 (+ 159us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.229090 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.229098 (+ 8us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:41.229207 (+ 109us) tablet.cc:701] Row locks acquired
0618 13:59:41.229211 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:41.229250 (+ 39us) write_op.cc:270] Start()
0618 13:59:41.229291 (+ 41us) write_op.cc:276] Timestamp: P: 1781791181229195 usec, L: 0
0618 13:59:41.229297 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.229506 (+ 209us) log.cc:844] Serialized 625 byte log entry
0618 13:59:41.233649 (+ 4143us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.233800 (+ 151us) write_op.cc:301] APPLY: starting
0618 13:59:41.233818 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.233886 (+ 68us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.233932 (+ 46us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.234050 (+ 118us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.234100 (+ 50us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:41.234107 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:41.234198 (+ 91us) log.cc:844] Serialized 29 byte log entry
0618 13:59:41.234274 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.234294 (+ 20us) write_op.cc:454] Released schema lock
0618 13:59:41.234300 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:41.234366 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.234370 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":99,"apply.run_cpu_time_us":559,"apply.run_wall_time_us":558,"num_ops":2,"prepare.queue_time_us":60,"prepare.run_cpu_time_us":900,"prepare.run_wall_time_us":901,"raft.queue_time_us":191,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":287,"replication_time_us":4321}]]}
I20260618 13:59:41.286697 29019 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 111) took 6534 ms. Trace:
I20260618 13:59:41.286906 29019 rpcz_store.cc:276] 0618 13:59:34.738147 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.066213 (+6328066us) service_pool.cc:224] Handling call
0618 13:59:41.066752 (+ 539us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:41.066923 (+ 171us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:41.067004 (+ 81us) catalog_manager.cc:2849] Locking tablets
0618 13:59:41.226462 (+159458us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:41.247072 (+ 20610us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:41.258400 (+ 11328us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:41.258497 (+ 97us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:41.272777 (+ 14280us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.238048 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.238153 (+ 105us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.238158 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:41.238160 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:41.238271 (+ 111us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.238282 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.238286 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:41.238383 (+ 97us) tablet.cc:701] Row locks acquired
0618 13:59:41.238385 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:41.238449 (+ 64us) write_op.cc:270] Start()
0618 13:59:41.238478 (+ 29us) write_op.cc:276] Timestamp: P: 1781791181238364 usec, L: 0
0618 13:59:41.238482 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.238638 (+ 156us) log.cc:844] Serialized 759 byte log entry
0618 13:59:41.246233 (+ 7595us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.246470 (+ 237us) write_op.cc:301] APPLY: starting
0618 13:59:41.246484 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.246523 (+ 39us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.246559 (+ 36us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.246624 (+ 65us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.246651 (+ 27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:41.246658 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:41.246723 (+ 65us) log.cc:844] Serialized 29 byte log entry
0618 13:59:41.246780 (+ 57us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.246799 (+ 19us) write_op.cc:454] Released schema lock
0618 13:59:41.246823 (+ 24us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:41.246925 (+ 102us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.246931 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":11935,"child_traces":[["op",{"apply.queue_time_us":180,"apply.run_cpu_time_us":438,"apply.run_wall_time_us":437,"num_ops":2,"prepare.queue_time_us":44,"prepare.run_cpu_time_us":675,"prepare.run_wall_time_us":682,"raft.queue_time_us":445,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":7,"replication_time_us":7720}]]}
I20260618 13:59:41.288303 29019 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_11c4074a927242a480bca5668deafc83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:41.289016 29019 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11c4074a927242a480bca5668deafc83 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:41.289323 29746 log.cc:1199] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/591465e2b57247a2b224cc8b09225811
I20260618 13:59:41.290102 29746 ts_tablet_manager.cc:1950] T 591465e2b57247a2b224cc8b09225811 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:41.294485 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet ba2d6894bf5649fc8cfae7e033c432b9 with delete_type TABLET_DATA_DELETED (replaced by tablet a6ba4976bb6e4c2da3a563ccad4eb2b0) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.297060 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f695a7e6a925470787a708c365e6b86b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.297351 30151 raft_consensus.cc:493] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.297425 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 141) took 6300 ms. Trace:
I20260618 13:59:41.297489 30151 raft_consensus.cc:515] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.297505 29746 rpcz_store.cc:276] 0618 13:59:34.997288 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.997693 (+ 405us) service_pool.cc:224] Handling call
0618 13:59:41.061795 (+6064102us) ts_tablet_manager.cc:1151] Deleting tablet 591465e2b57247a2b224cc8b09225811
0618 13:59:41.061804 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.113779 (+ 51975us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.221395 (+107616us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.297382 (+ 75987us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4732,"tablet-delete.queue_time_us":6005449}
I20260618 13:59:41.297840 28799 tablet_bootstrap.cc:492] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:41.298134 30151 leader_election.cc:290] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.298626 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 72c04f55890648f0a8184f3783243f7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:41.298799 29746 tablet_replica.cc:333] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:41.298862 30618 tablet_replica.cc:333] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:41.298993 29746 raft_consensus.cc:2243] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:41.299067 30618 raft_consensus.cc:2243] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:41.299144 29746 raft_consensus.cc:2272] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.299194 30618 raft_consensus.cc:2272] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.299412 30151 raft_consensus.cc:493] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.299624 28799 tablet_bootstrap.cc:654] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.299548 30151 raft_consensus.cc:515] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.299680 30618 ts_tablet_manager.cc:1916] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.300230 30151 leader_election.cc:290] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.301177 29746 ts_tablet_manager.cc:1916] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.304894 28537 raft_consensus.cc:3060] T f9f7359c4a144ca89fe3e51d1365a425 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.306685 30618 ts_tablet_manager.cc:1929] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:41.306771 30618 log.cc:1199] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/72c04f55890648f0a8184f3783243f7c
I20260618 13:59:41.307071 30618 ts_tablet_manager.cc:1950] T 72c04f55890648f0a8184f3783243f7c P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:41.308511 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet f2b21dcf74fa4264947ca43cc0026e76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:40 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:41.309643 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet ba2d6894bf5649fc8cfae7e033c432b9 with delete_type TABLET_DATA_DELETED (replaced by tablet a6ba4976bb6e4c2da3a563ccad4eb2b0) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:41.309928 30618 tablet_replica.cc:333] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:41.310217 30618 raft_consensus.cc:2243] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:41.310498 30618 raft_consensus.cc:2272] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.311056 30618 ts_tablet_manager.cc:1916] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:41.314553 28346 leader_election.cc:343] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:41.314837 28346 leader_election.cc:343] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.315093 28346 leader_election.cc:304] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
W20260618 13:59:41.286545 29017 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.3:42065 (request call id 147) took 2724 ms (client timeout 1000 ms). Trace:
I20260618 13:59:41.316924 30262 raft_consensus.cc:2804] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:41.317011 30262 raft_consensus.cc:493] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:41.317099 30262 raft_consensus.cc:3060] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:41.317466 28346 leader_election.cc:343] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.317489 30623 consensus_queue.cc:1048] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:41.317998 28346 leader_election.cc:304] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.318444 30623 raft_consensus.cc:2749] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:41.319121 28798 tablet_bootstrap.cc:492] T 7766014ee9814f01ae7ac3e849ca3d50 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:41.319315 28798 ts_tablet_manager.cc:1403] T 7766014ee9814f01ae7ac3e849ca3d50 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.164s user 0.032s sys 0.038s
I20260618 13:59:41.320073 28798 raft_consensus.cc:359] T 7766014ee9814f01ae7ac3e849ca3d50 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.320307 28798 raft_consensus.cc:385] T 7766014ee9814f01ae7ac3e849ca3d50 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.320433 28798 raft_consensus.cc:740] T 7766014ee9814f01ae7ac3e849ca3d50 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.320631 30262 raft_consensus.cc:515] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.320744 28798 consensus_queue.cc:260] T 7766014ee9814f01ae7ac3e849ca3d50 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.321197 30262 leader_election.cc:290] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.321637 28798 ts_tablet_manager.cc:1434] T 7766014ee9814f01ae7ac3e849ca3d50 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:41.321996 28798 tablet_bootstrap.cc:492] T 2d1073b726ef4cbeac6c874fc8863e90 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:41.322712 29026 master_service.cc:960] ReplaceTablet: received request to replace tablet b0d3d363766546c4a97b0763bad11f71 from {username='slave'} at 127.0.0.1:33144
W20260618 13:59:41.323167 28346 leader_election.cc:343] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.272251 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b59255b14df4158afe37cb8890e7415 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:41.323233 29017 rpcz_store.cc:269] 0618 13:59:38.547045 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.486408 (+939363us) service_pool.cc:224] Handling call
0618 13:59:41.271897 (+1785489us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:40.208334 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:40.208514 (+ 180us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:40.208525 (+ 11us) write_op.cc:435] Acquired schema lock
0618 13:59:40.208529 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:40.208975 (+ 446us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:40.208996 (+ 21us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:40.209004 (+ 8us) tablet.cc:685] Acquiring locks for 11 operations
0618 13:59:40.209431 (+ 427us) tablet.cc:701] Row locks acquired
0618 13:59:40.209435 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:40.209497 (+ 62us) write_op.cc:270] Start()
0618 13:59:40.209543 (+ 46us) write_op.cc:276] Timestamp: P: 1781791180209435 usec, L: 0
0618 13:59:40.209550 (+ 7us) op_driver.cc:348] REPLICATION: starting
0618 13:59:40.209826 (+ 276us) log.cc:844] Serialized 4301 byte log entry
0618 13:59:40.214827 (+ 5001us) op_driver.cc:464] REPLICATION: finished
0618 13:59:40.214932 (+ 105us) write_op.cc:301] APPLY: starting
0618 13:59:40.214949 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:40.215056 (+ 107us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:40.215122 (+ 66us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:40.215327 (+ 205us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:40.215377 (+ 50us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0618 13:59:40.215384 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:40.215484 (+ 100us) log.cc:844] Serialized 83 byte log entry
0618 13:59:40.215554 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:40.215585 (+ 31us) write_op.cc:454] Released schema lock
0618 13:59:40.215592 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:40.215664 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:40.215667 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":111316,"reported_tablets":190,"child_traces":[["op",{"apply.queue_time_us":57,"apply.run_cpu_time_us":720,"apply.run_wall_time_us":719,"num_ops":11,"prepare.queue_time_us":75,"prepare.run_cpu_time_us":1632,"prepare.run_wall_time_us":1632,"raft.queue_time_us":598,"raft.run_cpu_time_us":394,"raft.run_wall_time_us":393,"replication_time_us":5245}]]}
I20260618 13:59:41.323735 28657 tablet_service.cc:1558] Processing DeleteTablet for tablet ba2d6894bf5649fc8cfae7e033c432b9 with delete_type TABLET_DATA_DELETED (replaced by tablet a6ba4976bb6e4c2da3a563ccad4eb2b0) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.272714 28655 tablet_service.cc:1558] Processing DeleteTablet for tablet 18d8b609b96a4e81b57f69bff522eeea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.273093 28654 tablet_service.cc:1558] Processing DeleteTablet for tablet b51d0851efb34eaa9e9e4be6b292ef62 with delete_type TABLET_DATA_DELETED (replaced by tablet f9f7359c4a144ca89fe3e51d1365a425) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.276018 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be59f988f0e34b9480f859cf88b9c223" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.282613 28652 tablet_service.cc:1558] Processing DeleteTablet for tablet fddea8fba4da40d7a5780e6750caf5e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.282696 28656 tablet_service.cc:1558] Processing DeleteTablet for tablet 37a6764a5e024903848561c9935b1a43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.282831 28662 tablet_service.cc:1558] Processing DeleteTablet for tablet dd2385e6752d49429422fc854539e9c6 with delete_type TABLET_DATA_DELETED (replaced by tablet c38a2156a55e4e5eb91d101e9a61eecd) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.282948 28661 tablet_service.cc:1558] Processing DeleteTablet for tablet f89c9954d48e407c96c294c26644e25c with delete_type TABLET_DATA_DELETED (replaced by tablet b0d3d363766546c4a97b0763bad11f71) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.283154 28647 tablet_service.cc:1558] Processing DeleteTablet for tablet 36b8a46a41e543c58a946b6764618c7d with delete_type TABLET_DATA_DELETED (replaced by tablet f89c9954d48e407c96c294c26644e25c) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.283206 28659 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.324497 28484 leader_election.cc:304] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.283354 28663 tablet_service.cc:1558] Processing DeleteTablet for tablet d1baf86497d442dbbbddbc3498bebbee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.283455 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet 7aa9f65d8a5c46fdb745ee17af946dce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.283565 28649 tablet_service.cc:1558] Processing DeleteTablet for tablet e95c88e4b0a046a3ad69ee49af47c7c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.285238 28658 tablet_service.cc:1558] Processing DeleteTablet for tablet d502d67826ff4d7a8b0f515e5d8b6d77 with delete_type TABLET_DATA_DELETED (replaced by tablet 657e55fa53c447babfe91b1582806e6b) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.285398 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet bce7bf74b6d24d0bb43808b8a62cf1b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.324869 30551 raft_consensus.cc:2749] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:41.285559 28651 tablet_service.cc:1558] Processing DeleteTablet for tablet b0285c75c632457383e81805ab036aaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.285682 28648 tablet_service.cc:1558] Processing DeleteTablet for tablet fe750fc5d998448caeab0a6c808dfba7 with delete_type TABLET_DATA_DELETED (replaced by tablet 438663f365e34fab87e34f434920cecf) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.285810 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet 72c04f55890648f0a8184f3783243f7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.291973 28646 tablet_service.cc:1558] Processing DeleteTablet for tablet d7a97176f72349cda221baaea61d18ad with delete_type TABLET_DATA_DELETED (replaced by tablet 72fd63e8245d4042b091593881ba11b6) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.292511 28644 tablet_service.cc:1558] Processing DeleteTablet for tablet 751cd9bd961b4f56a588ed02a05f0212 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.325153 28346 leader_election.cc:304] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.294370 28645 tablet_service.cc:1558] Processing DeleteTablet for tablet d667ea5f029f46d89fcb3be03069b97b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.325431 30152 raft_consensus.cc:2749] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:41.325489 28346 leader_election.cc:343] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:41.325703 28346 leader_election.cc:343] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:41.325845 28346 leader_election.cc:343] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.326292 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffd607dd9eb43e0a3769fcdd28ed05f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.327724 28798 tablet_bootstrap.cc:654] T 2d1073b726ef4cbeac6c874fc8863e90 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.328203 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffd607dd9eb43e0a3769fcdd28ed05f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.295145 28797 tablet_bootstrap.cc:492] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:41.296274 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08132883fb4f4632af2961bead244b18" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.297142 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f695a7e6a925470787a708c365e6b86b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.329231 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c50ab60a26647558da509236aac1e8b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.329473 28797 ts_tablet_manager.cc:1403] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.107s user 0.000s sys 0.003s
I20260618 13:59:41.330863 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c50ab60a26647558da509236aac1e8b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.332166 28797 raft_consensus.cc:359] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.332582 28797 raft_consensus.cc:385] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.332669 28797 raft_consensus.cc:740] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.332891 28797 consensus_queue.cc:260] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.333456 28797 ts_tablet_manager.cc:1434] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260618 13:59:41.333452 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 72c04f55890648f0a8184f3783243f7c (table default.Table_773be314d7e741fcb4fb6eb7d954802e [id=0d83fc68255c458587390c58b12e00d7]) successfully deleted
I20260618 13:59:41.333722 28797 tablet_bootstrap.cc:492] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:41.333729 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet f2b21dcf74fa4264947ca43cc0026e76 because the tablet was not found. No further retry: Not found: Tablet not found: f2b21dcf74fa4264947ca43cc0026e76
I20260618 13:59:41.335268 28797 tablet_bootstrap.cc:654] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.340042 28545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed648dcf882340388155068d7a5a7869" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:41.340411 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed648dcf882340388155068d7a5a7869" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:41.340564 28683 raft_consensus.cc:3060] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.343607 28683 raft_consensus.cc:2468] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
W20260618 13:59:41.344435 28346 leader_election.cc:343] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:41.348071 28346 leader_election.cc:343] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.348664 28346 leader_election.cc:304] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.349195 30262 raft_consensus.cc:2749] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:41.349327 28346 leader_election.cc:343] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.349442 28346 leader_election.cc:304] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.349756 30262 raft_consensus.cc:2749] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:41.349752 28346 leader_election.cc:343] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.351580 28346 leader_election.cc:304] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:41.351787 30262 raft_consensus.cc:2804] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:41.351934 30262 raft_consensus.cc:697] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:41.352192 30262 consensus_queue.cc:237] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:41.352416 29024 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.1:48495 (request call id 150) took 1371 ms (client timeout 1000 ms). Trace:
W20260618 13:59:41.351788 28346 leader_election.cc:343] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.352682 28346 leader_election.cc:304] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:41.352988 28346 leader_election.cc:343] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:41.352536 29024 rpcz_store.cc:269] 0618 13:59:39.980493 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.249407 (+268914us) service_pool.cc:224] Handling call
0618 13:59:41.352352 (+1102945us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:40.696224 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:40.696365 (+ 141us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:40.696374 (+ 9us) write_op.cc:435] Acquired schema lock
0618 13:59:40.696379 (+ 5us) tablet.cc:662] Decoding operations
0618 13:59:40.696696 (+ 317us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:40.696716 (+ 20us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:40.696724 (+ 8us) tablet.cc:685] Acquiring locks for 7 operations
0618 13:59:40.696898 (+ 174us) tablet.cc:701] Row locks acquired
0618 13:59:40.696901 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:40.696937 (+ 36us) write_op.cc:270] Start()
0618 13:59:40.696990 (+ 53us) write_op.cc:276] Timestamp: P: 1781791180696882 usec, L: 0
0618 13:59:40.696997 (+ 7us) op_driver.cc:348] REPLICATION: starting
0618 13:59:40.697206 (+ 209us) log.cc:844] Serialized 2793 byte log entry
0618 13:59:40.700630 (+ 3424us) op_driver.cc:464] REPLICATION: finished
0618 13:59:40.700763 (+ 133us) write_op.cc:301] APPLY: starting
0618 13:59:40.700777 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:40.700865 (+ 88us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:40.700941 (+ 76us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:40.701166 (+ 225us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:40.701221 (+ 55us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0618 13:59:40.701233 (+ 12us) write_op.cc:312] APPLY: finished
0618 13:59:40.701375 (+ 142us) log.cc:844] Serialized 59 byte log entry
0618 13:59:40.701460 (+ 85us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:40.701495 (+ 35us) write_op.cc:454] Released schema lock
0618 13:59:40.701502 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:40.701583 (+ 81us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:40.701589 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":140,"child_traces":[["op",{"apply.queue_time_us":73,"apply.run_cpu_time_us":803,"apply.run_wall_time_us":803,"num_ops":7,"prepare.queue_time_us":217,"prepare.run_cpu_time_us":1167,"prepare.run_wall_time_us":1168,"raft.queue_time_us":125,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":255,"replication_time_us":3605}]]}
I20260618 13:59:41.353073 28346 leader_election.cc:304] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.353109 30152 raft_consensus.cc:2749] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:41.353241 30152 raft_consensus.cc:2749] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:41.353387 28346 leader_election.cc:343] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.358515 30551 raft_consensus.cc:493] T 538077e459a44ed78c95f2e881b6e0b0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.358700 30551 raft_consensus.cc:515] T 538077e459a44ed78c95f2e881b6e0b0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.359453 30551 leader_election.cc:290] T 538077e459a44ed78c95f2e881b6e0b0 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.359915 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "538077e459a44ed78c95f2e881b6e0b0" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.359959 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "538077e459a44ed78c95f2e881b6e0b0" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:41.360468 28483 leader_election.cc:304] T 538077e459a44ed78c95f2e881b6e0b0 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.360832 30551 raft_consensus.cc:2749] T 538077e459a44ed78c95f2e881b6e0b0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:41.347595 30618 ts_tablet_manager.cc:1929] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:41.362581 30618 log.cc:1199] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/ba2d6894bf5649fc8cfae7e033c432b9
I20260618 13:59:41.363312 30618 ts_tablet_manager.cc:1950] T ba2d6894bf5649fc8cfae7e033c432b9 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:41.364394 28797 tablet_bootstrap.cc:492] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:41.366364 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet ba2d6894bf5649fc8cfae7e033c432b9 (table default.Table_dba55442642c45b1b8dfe959d62699c9 [id=596cac3f4f6540019b75bb586ec7f9ab]) successfully deleted
I20260618 13:59:41.367182 28798 tablet_bootstrap.cc:492] T 2d1073b726ef4cbeac6c874fc8863e90 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:41.367370 28798 ts_tablet_manager.cc:1403] T 2d1073b726ef4cbeac6c874fc8863e90 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.045s user 0.005s sys 0.000s
I20260618 13:59:41.368301 28798 raft_consensus.cc:359] T 2d1073b726ef4cbeac6c874fc8863e90 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.368532 28798 raft_consensus.cc:385] T 2d1073b726ef4cbeac6c874fc8863e90 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.368625 28798 raft_consensus.cc:740] T 2d1073b726ef4cbeac6c874fc8863e90 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.368861 28798 consensus_queue.cc:260] T 2d1073b726ef4cbeac6c874fc8863e90 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.368865 29017 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_7a3559b2cbe044d3b44cf3961c7d2d51"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:41.364533 28797 ts_tablet_manager.cc:1403] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.000s
W20260618 13:59:41.369709 29017 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a3559b2cbe044d3b44cf3961c7d2d51 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:41.370042 28797 raft_consensus.cc:359] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.370193 28797 raft_consensus.cc:385] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.370250 28797 raft_consensus.cc:740] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.371554 30586 raft_consensus.cc:493] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.371709 30586 raft_consensus.cc:515] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.373499 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f7b66652f464893ad812d27e883a9a4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:41.373677 28403 raft_consensus.cc:2468] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:41.372710 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f7b66652f464893ad812d27e883a9a4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.374047 28617 leader_election.cc:304] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:41.374260 30572 raft_consensus.cc:2804] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:41.374315 30572 raft_consensus.cc:493] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:41.374364 30572 raft_consensus.cc:3060] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:41.374472 28618 leader_election.cc:343] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.375172 28798 ts_tablet_manager.cc:1434] T 2d1073b726ef4cbeac6c874fc8863e90 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260618 13:59:41.375455 28798 tablet_bootstrap.cc:492] T 80e99a85e83c46afa367e59e5ddb7dc7 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
W20260618 13:59:41.375959 29020 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.3:42065 (request call id 144) took 6034 ms (client timeout 1000 ms). Trace:
I20260618 13:59:41.376884 30572 raft_consensus.cc:515] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.377408 30572 leader_election.cc:290] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:41.377748 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f7b66652f464893ad812d27e883a9a4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:41.378121 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f7b66652f464893ad812d27e883a9a4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
W20260618 13:59:41.378192 28618 leader_election.cc:343] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.378252 28403 raft_consensus.cc:3060] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.380900 28403 raft_consensus.cc:2468] T 8f7b66652f464893ad812d27e883a9a4 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:41.381111 28798 tablet_bootstrap.cc:654] T 80e99a85e83c46afa367e59e5ddb7dc7 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.381283 28617 leader_election.cc:304] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:41.382164 28644 tablet_service.cc:1558] Processing DeleteTablet for tablet 0463fa2e30554abcbfc8f37b702e81d4 with delete_type TABLET_DATA_DELETED (replaced by tablet fe1139c409e84070a2f496a85545e8d5) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.383806 30572 raft_consensus.cc:2804] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:41.384649 30572 raft_consensus.cc:697] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:41.384924 30572 consensus_queue.cc:237] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.387697 30586 leader_election.cc:290] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:41.389886 28797 consensus_queue.cc:260] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.390085 30572 raft_consensus.cc:493] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.390187 30572 raft_consensus.cc:515] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.391238 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f695a7e6a925470787a708c365e6b86b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:41.391263 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f695a7e6a925470787a708c365e6b86b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.391424 28403 raft_consensus.cc:2468] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:41.391788 28617 leader_election.cc:304] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
W20260618 13:59:41.392211 28618 leader_election.cc:343] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.392292 29303 raft_consensus.cc:2804] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:41.392364 29303 raft_consensus.cc:493] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:41.392421 29303 raft_consensus.cc:3060] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.395031 28799 tablet_bootstrap.cc:492] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:41.395341 29303 raft_consensus.cc:515] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.395900 29303 leader_election.cc:290] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:41.396545 30152 raft_consensus.cc:493] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.396672 30152 raft_consensus.cc:515] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.396900 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 591465e2b57247a2b224cc8b09225811 (table default.Table_0a85d79af65f48a3803bfd1248593a11 [id=2f4d62be772a421ab777fd23be58b719]) successfully deleted
I20260618 13:59:41.397238 30152 leader_election.cc:290] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.397434 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f695a7e6a925470787a708c365e6b86b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:41.397495 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f695a7e6a925470787a708c365e6b86b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:41.397639 28403 raft_consensus.cc:3060] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.397992 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8f124ba79b46e098303b7b2705844d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.398118 28799 ts_tablet_manager.cc:1403] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.100s user 0.008s sys 0.003s
W20260618 13:59:41.398548 28346 leader_election.cc:343] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.399009 28799 raft_consensus.cc:359] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.399212 28799 raft_consensus.cc:385] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.399269 28799 raft_consensus.cc:740] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.399537 28799 consensus_queue.cc:260] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.400146 28799 ts_tablet_manager.cc:1434] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:41.400444 28799 tablet_bootstrap.cc:492] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:41.400504 28403 raft_consensus.cc:2468] T f695a7e6a925470787a708c365e6b86b P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:41.400866 28617 leader_election.cc:304] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:41.401085 29303 raft_consensus.cc:2804] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:41.401180 29303 raft_consensus.cc:697] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:41.401546 29303 consensus_queue.cc:237] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.402130 28799 tablet_bootstrap.cc:654] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.397619 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8f124ba79b46e098303b7b2705844d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:41.403054 28346 leader_election.cc:343] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.403155 28346 leader_election.cc:304] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:41.403173 28618 leader_election.cc:343] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.403592 30572 leader_election.cc:290] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:41.405691 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 76f83f23f22345bb9bc3f239642ea673 (DEFAULT_TABLE table=default.Table_6024ed5a738d4f91b734fa235807f43f [id=e5d482a2aa664d108c7d829ac2c5e469]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:41.406047 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76f83f23f22345bb9bc3f239642ea673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:41.410508 30623 raft_consensus.cc:2749] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:41.416857 28797 ts_tablet_manager.cc:1434] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.047s user 0.001s sys 0.000s
I20260618 13:59:41.422778 29746 ts_tablet_manager.cc:1929] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:41.423008 30551 raft_consensus.cc:493] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6fc808af0e884ed8836561665d8e0e7b)
I20260618 13:59:41.423161 30551 raft_consensus.cc:515] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:41.423708 28528 tablet_service.cc:1511] Processing CreateTablet for tablet 76f83f23f22345bb9bc3f239642ea673 (DEFAULT_TABLE table=default.Table_6024ed5a738d4f91b734fa235807f43f [id=e5d482a2aa664d108c7d829ac2c5e469]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:41.424136 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76f83f23f22345bb9bc3f239642ea673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:41.423712 30551 leader_election.cc:290] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:41.424960 28484 leader_election.cc:343] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 0463fa2e30554abcbfc8f37b702e81d4
W20260618 13:59:41.425441 28483 leader_election.cc:343] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 0463fa2e30554abcbfc8f37b702e81d4
I20260618 13:59:41.425547 28483 leader_election.cc:304] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.425889 30551 raft_consensus.cc:2749] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:41.428170 28644 tablet_service.cc:1511] Processing CreateTablet for tablet 76f83f23f22345bb9bc3f239642ea673 (DEFAULT_TABLE table=default.Table_6024ed5a738d4f91b734fa235807f43f [id=e5d482a2aa664d108c7d829ac2c5e469]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:41.428534 28644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76f83f23f22345bb9bc3f239642ea673. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:41.429759 28618 consensus_peers.cc:597] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:41.427135 28797 tablet_bootstrap.cc:492] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:41.431170 28797 tablet_bootstrap.cc:654] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.433527 29303 raft_consensus.cc:493] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6fc808af0e884ed8836561665d8e0e7b)
I20260618 13:59:41.433640 29303 raft_consensus.cc:515] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:41.436105 29566 ts_tablet_manager.cc:1929] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:41.436528 30262 raft_consensus.cc:493] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.436669 30262 raft_consensus.cc:515] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.437216 30262 leader_election.cc:290] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.438141 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bbb71449d434d54b007d6f0da0796d7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.437898 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bbb71449d434d54b007d6f0da0796d7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:41.438643 28346 leader_election.cc:343] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.441184 29303 leader_election.cc:290] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:41.442991 28346 leader_election.cc:343] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.443109 28346 leader_election.cc:304] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.443791 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be59f988f0e34b9480f859cf88b9c223" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.443979 28537 raft_consensus.cc:2468] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:41.447221 28618 leader_election.cc:304] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:41.447930 30584 raft_consensus.cc:2804] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260618 13:59:41.448030 30584 raft_consensus.cc:493] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6fc808af0e884ed8836561665d8e0e7b)
I20260618 13:59:41.448101 30584 raft_consensus.cc:3060] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Advancing to term 2
I20260618 13:59:41.448493 29019 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 115) took 5893 ms. Trace:
I20260618 13:59:41.448585 29019 rpcz_store.cc:276] 0618 13:59:35.554787 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.287807 (+5733020us) service_pool.cc:224] Handling call
0618 13:59:41.289149 (+ 1342us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:41.289736 (+ 587us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:41.448206 (+158470us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:41.448341 (+ 135us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:41.448434 (+ 93us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.290451 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.290551 (+ 100us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.290558 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:41.290560 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:41.290653 (+ 93us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.290665 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.290672 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:41.290746 (+ 74us) tablet.cc:701] Row locks acquired
0618 13:59:41.290748 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:41.290785 (+ 37us) write_op.cc:270] Start()
0618 13:59:41.290821 (+ 36us) write_op.cc:276] Timestamp: P: 1781791181290724 usec, L: 0
0618 13:59:41.290825 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.291021 (+ 196us) log.cc:844] Serialized 433 byte log entry
0618 13:59:41.442928 (+151907us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.443314 (+ 386us) write_op.cc:301] APPLY: starting
0618 13:59:41.443340 (+ 26us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.443387 (+ 47us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.443422 (+ 35us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.443548 (+ 126us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.443602 (+ 54us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:41.443613 (+ 11us) write_op.cc:312] APPLY: finished
0618 13:59:41.443706 (+ 93us) log.cc:844] Serialized 29 byte log entry
0618 13:59:41.443774 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.443797 (+ 23us) write_op.cc:454] Released schema lock
0618 13:59:41.443806 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:41.443923 (+ 117us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.443928 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":311,"apply.run_cpu_time_us":580,"apply.run_wall_time_us":580,"num_ops":2,"prepare.queue_time_us":62,"prepare.run_cpu_time_us":742,"prepare.run_wall_time_us":1132,"raft.queue_time_us":902,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":8,"replication_time_us":152058}]]}
I20260618 13:59:41.449937 29019 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_34210bd1a3164a189c219c1ed1baeb97"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:41.450677 28617 leader_election.cc:343] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: be59f988f0e34b9480f859cf88b9c223
I20260618 13:59:41.450626 30584 raft_consensus.cc:515] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:41.450842 30262 raft_consensus.cc:2749] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:41.451092 29566 log.cc:1199] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/e6de1293572f4e75af3417f281b9a142
I20260618 13:59:41.451424 29566 ts_tablet_manager.cc:1950] T e6de1293572f4e75af3417f281b9a142 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:41.452095 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 110) took 9302 ms. Trace:
I20260618 13:59:41.452173 29566 rpcz_store.cc:276] 0618 13:59:32.149453 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.149495 (+ 42us) service_pool.cc:224] Handling call
0618 13:59:41.233635 (+9084140us) ts_tablet_manager.cc:1151] Deleting tablet e6de1293572f4e75af3417f281b9a142
0618 13:59:41.233645 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.364756 (+131111us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.436057 (+ 71301us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.452079 (+ 16022us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9083970}
I20260618 13:59:41.451152 30584 leader_election.cc:290] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:41.453331 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be59f988f0e34b9480f859cf88b9c223" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:41.453526 28537 raft_consensus.cc:3060] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Advancing to term 2
I20260618 13:59:41.456185 28537 raft_consensus.cc:2468] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 2.
W20260618 13:59:41.456357 29019 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_34210bd1a3164a189c219c1ed1baeb97 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:41.457921 28537 raft_consensus.cc:3060] T 538077e459a44ed78c95f2e881b6e0b0 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:41.455310 28617 leader_election.cc:343] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: be59f988f0e34b9480f859cf88b9c223
I20260618 13:59:41.458380 28618 leader_election.cc:304] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:41.458832 29026 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet b0d3d363766546c4a97b0763bad11f71
I20260618 13:59:41.459017 30586 raft_consensus.cc:2804] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [term 2 FOLLOWER]: Leader election won for term 2
I20260618 13:59:41.459122 30586 raft_consensus.cc:697] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [term 2 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:41.459367 30586 consensus_queue.cc:237] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } }
I20260618 13:59:41.459540 29026 catalog_manager.cc:6784] ReplaceTablet: tablet b0d3d363766546c4a97b0763bad11f71 deleted and replaced by tablet 00b602a473a94c05a617fb9478300c74
I20260618 13:59:41.459651 29026 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 114) took 5923 ms. Trace:
I20260618 13:59:41.460041 29566 tablet_replica.cc:333] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:41.459743 29026 rpcz_store.cc:276] 0618 13:59:35.536285 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.286103 (+5749818us) service_pool.cc:224] Handling call
0618 13:59:41.459619 (+173516us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.341319 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.341406 (+ 87us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.341413 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:41.341415 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:41.341522 (+ 107us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.341534 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.341538 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:41.341614 (+ 76us) tablet.cc:701] Row locks acquired
0618 13:59:41.341616 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:41.341648 (+ 32us) write_op.cc:270] Start()
0618 13:59:41.341682 (+ 34us) write_op.cc:276] Timestamp: P: 1781791181341584 usec, L: 0
0618 13:59:41.341687 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.341860 (+ 173us) log.cc:844] Serialized 659 byte log entry
0618 13:59:41.443019 (+101159us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.443440 (+ 421us) write_op.cc:301] APPLY: starting
0618 13:59:41.443450 (+ 10us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.443472 (+ 22us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.445855 (+ 2383us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.446014 (+ 159us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.446064 (+ 50us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:41.446071 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:41.446129 (+ 58us) log.cc:844] Serialized 29 byte log entry
0618 13:59:41.446180 (+ 51us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.446196 (+ 16us) write_op.cc:454] Released schema lock
0618 13:59:41.446201 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:41.446262 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.446266 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":367,"apply.run_cpu_time_us":507,"apply.run_wall_time_us":2814,"num_ops":2,"prepare.queue_time_us":51,"prepare.run_cpu_time_us":646,"prepare.run_wall_time_us":647,"raft.queue_time_us":430,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":7,"replication_time_us":101309,"thread_start_us":1224,"threads_started":1}]]}
I20260618 13:59:41.460228 29566 raft_consensus.cc:2243] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:41.460285 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet b0d3d363766546c4a97b0763bad11f71 with delete_type TABLET_DATA_DELETED (replaced by tablet 00b602a473a94c05a617fb9478300c74) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:41.460480 29566 raft_consensus.cc:2272] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.460567 30618 tablet_replica.cc:333] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:41.460749 30618 raft_consensus.cc:2243] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:41.460908 30618 raft_consensus.cc:2272] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.460943 29566 ts_tablet_manager.cc:1916] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.461083 29026 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_982b749682bb4ee8ba1b7a9c4c897f08"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:41.461681 29026 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_982b749682bb4ee8ba1b7a9c4c897f08 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:41.462073 30618 ts_tablet_manager.cc:1916] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.463935 29017 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 116) took 5907 ms. Trace:
I20260618 13:59:41.464046 29017 rpcz_store.cc:276] 0618 13:59:35.556232 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.324239 (+5768007us) service_pool.cc:224] Handling call
0618 13:59:41.368746 (+ 44507us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb46846c3 00007fcbab0e07c6 00007fcbab0e04f5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:41.370024 (+ 1278us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:41.382975 (+ 12951us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:41.463712 (+ 80737us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:41.463807 (+ 95us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:41.463896 (+ 89us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.395612 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.395747 (+ 135us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.395754 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:41.395757 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:41.395876 (+ 119us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.395889 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.395895 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:41.396024 (+ 129us) tablet.cc:701] Row locks acquired
0618 13:59:41.396028 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:41.396076 (+ 48us) write_op.cc:270] Start()
0618 13:59:41.396126 (+ 50us) write_op.cc:276] Timestamp: P: 1781791181396052 usec, L: 0
0618 13:59:41.396131 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.396306 (+ 175us) log.cc:844] Serialized 433 byte log entry
0618 13:59:41.444331 (+ 48025us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.444473 (+ 142us) write_op.cc:301] APPLY: starting
0618 13:59:41.444487 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.444535 (+ 48us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.444563 (+ 28us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.444671 (+ 108us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.444706 (+ 35us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:41.444715 (+ 9us) write_op.cc:312] APPLY: finished
0618 13:59:41.444782 (+ 67us) log.cc:844] Serialized 29 byte log entry
0618 13:59:41.444835 (+ 53us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.444861 (+ 26us) write_op.cc:454] Released schema lock
0618 13:59:41.444867 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:41.444931 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.444938 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":44953856,"child_traces":[["op",{"apply.queue_time_us":101,"apply.run_cpu_time_us":450,"apply.run_wall_time_us":450,"num_ops":2,"prepare.queue_time_us":12128,"prepare.run_cpu_time_us":798,"prepare.run_wall_time_us":798,"raft.queue_time_us":338,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":6,"replication_time_us":48177}]]}
I20260618 13:59:41.465451 29017 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_41cc80170f054b5c962ef36e7c322fce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:41.465970 29017 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_41cc80170f054b5c962ef36e7c322fce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:41.466562 30577 raft_consensus.cc:493] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6fc808af0e884ed8836561665d8e0e7b)
I20260618 13:59:41.466676 30577 raft_consensus.cc:515] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.467183 30577 leader_election.cc:290] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
W20260618 13:59:41.376068 29020 rpcz_store.cc:269] 0618 13:59:35.341685 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.357473 (+ 15788us) service_pool.cc:224] Handling call
0618 13:59:35.429619 (+ 72146us) spinlock_profiling.cc:243] Waited 57.8 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb467ef08 00007fcbab0ddca6 00007fcbab0dd9d5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:41.375921 (+5946302us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:36.779051 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:36.779204 (+ 153us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:36.779213 (+ 9us) write_op.cc:435] Acquired schema lock
0618 13:59:36.779217 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:36.779473 (+ 256us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:36.779490 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:36.779498 (+ 8us) tablet.cc:685] Acquiring locks for 4 operations
0618 13:59:36.779644 (+ 146us) tablet.cc:701] Row locks acquired
0618 13:59:36.779648 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:36.779688 (+ 40us) write_op.cc:270] Start()
0618 13:59:36.779727 (+ 39us) write_op.cc:276] Timestamp: P: 1781791176779669 usec, L: 0
0618 13:59:36.779732 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:36.779949 (+ 217us) log.cc:844] Serialized 1662 byte log entry
0618 13:59:36.783863 (+ 3914us) op_driver.cc:464] REPLICATION: finished
0618 13:59:36.783985 (+ 122us) write_op.cc:301] APPLY: starting
0618 13:59:36.784002 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:36.784088 (+ 86us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:36.784113 (+ 25us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:36.784202 (+ 89us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:36.784236 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0618 13:59:36.784243 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:36.784341 (+ 98us) log.cc:844] Serialized 41 byte log entry
0618 13:59:36.784404 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.784427 (+ 23us) write_op.cc:454] Released schema lock
0618 13:59:36.784435 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:36.784508 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:36.784514 (+ 6us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:37.589403 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:37.619231 (+ 29828us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:37.619238 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:37.619240 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:37.619353 (+ 113us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:37.619366 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:37.619371 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:37.619442 (+ 71us) tablet.cc:701] Row locks acquired
0618 13:59:37.619444 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:37.619484 (+ 40us) write_op.cc:270] Start()
0618 13:59:37.619520 (+ 36us) write_op.cc:276] Timestamp: P: 1781791177619413 usec, L: 0
0618 13:59:37.619524 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:37.704439 (+ 84915us) log.cc:844] Serialized 356 byte log entry
0618 13:59:37.706944 (+ 2505us) spinlock_profiling.cc:243] Waited 84.7 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fcbac75ef22 00007fcbadb40305 00007fcbadb3ee29 00007fcbadb47250 00007fcbadb4705d 0000000000aa681c 00007fcba07299ff 00007fcba072cd1b 00007fcba072cb98 0000000000aa681c
0618 13:59:37.712617 (+ 5673us) op_driver.cc:464] REPLICATION: finished
0618 13:59:37.712803 (+ 186us) write_op.cc:301] APPLY: starting
0618 13:59:37.712819 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:37.712859 (+ 40us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:37.712862 (+ 3us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:37.712911 (+ 49us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:37.712941 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:37.712948 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:37.713020 (+ 72us) log.cc:844] Serialized 23 byte log entry
0618 13:59:37.713077 (+ 57us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:37.713096 (+ 19us) write_op.cc:454] Released schema lock
0618 13:59:37.713103 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:37.731435 (+ 18332us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:37.731443 (+ 8us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":40718,"reported_tablets":204,"spinlock_wait_cycles":133019520,"child_traces":[["op",{"apply.queue_time_us":73,"apply.run_cpu_time_us":514,"apply.run_wall_time_us":513,"num_ops":4,"prepare.queue_time_us":56,"prepare.run_cpu_time_us":1009,"prepare.run_wall_time_us":1009,"raft.queue_time_us":459,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":293,"replication_time_us":4101}],["op",{"apply.queue_time_us":134,"apply.run_cpu_time_us":357,"apply.run_wall_time_us":357,"num_ops":1,"prepare.queue_time_us":92829,"prepare.run_cpu_time_us":1168,"prepare.run_wall_time_us":142021,"raft.queue_time_us":2865,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":286,"replication_time_us":93059,"spinlock_wait_cycles":194806528}]]}
I20260618 13:59:41.472218 29020 master_service.cc:960] ReplaceTablet: received request to replace tablet ca2ee746c5024cfc9b31c470f9616e7d from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:41.472415 28797 tablet_bootstrap.cc:492] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:41.472573 28797 ts_tablet_manager.cc:1403] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.046s user 0.002s sys 0.000s
I20260618 13:59:41.472728 29020 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 120) took 5754 ms. Trace:
I20260618 13:59:41.472817 29020 rpcz_store.cc:276] 0618 13:59:35.718736 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.472143 (+5753407us) service_pool.cc:224] Handling call
0618 13:59:41.472677 (+ 534us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260618 13:59:41.473199 28797 raft_consensus.cc:359] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.473479 28797 raft_consensus.cc:385] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.473590 28797 raft_consensus.cc:740] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.473578 29020 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_38ca98ca393f4d549e1184885e55e64b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:41.473742 30577 consensus_queue.cc:1048] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:41.473927 28797 consensus_queue.cc:260] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:41.474215 29020 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_38ca98ca393f4d549e1184885e55e64b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:41.474682 28797 ts_tablet_manager.cc:1434] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:41.475939 28797 tablet_bootstrap.cc:492] T d0bcff62290c43c9be7cc496fbff6142 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:41.476645 28617 leader_election.cc:343] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: cea4bc13eabc4dd98b4f96723498014c
I20260618 13:59:41.476922 29024 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_e4086eb879a44971b5f69ef46e8c56c0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:41.477487 29024 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e4086eb879a44971b5f69ef46e8c56c0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:41.478011 28797 tablet_bootstrap.cc:654] T d0bcff62290c43c9be7cc496fbff6142 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.459682 29746 log.cc:1199] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/864f2602840c40fc8606b3784e671cba
I20260618 13:59:41.468649 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea4bc13eabc4dd98b4f96723498014c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.482286 28546 raft_consensus.cc:2468] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:41.482797 28618 leader_election.cc:304] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:41.483237 30584 raft_consensus.cc:2804] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260618 13:59:41.483377 30584 raft_consensus.cc:493] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6fc808af0e884ed8836561665d8e0e7b)
I20260618 13:59:41.483525 30584 raft_consensus.cc:3060] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Advancing to term 2
I20260618 13:59:41.485107 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet b0d3d363766546c4a97b0763bad11f71 with delete_type TABLET_DATA_DELETED (replaced by tablet 00b602a473a94c05a617fb9478300c74) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.485368 28644 tablet_service.cc:1558] Processing DeleteTablet for tablet b0d3d363766546c4a97b0763bad11f71 with delete_type TABLET_DATA_DELETED (replaced by tablet 00b602a473a94c05a617fb9478300c74) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.486763 29746 ts_tablet_manager.cc:1950] T 864f2602840c40fc8606b3784e671cba P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:41.487733 29566 ts_tablet_manager.cc:1929] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:41.487826 29566 log.cc:1199] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/04dac931e62a448d99633878eb70b92f
I20260618 13:59:41.492914 29566 ts_tablet_manager.cc:1950] T 04dac931e62a448d99633878eb70b92f P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:41.494747 30622 consensus_queue.cc:1048] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:41.496598 30262 raft_consensus.cc:493] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.496779 30262 raft_consensus.cc:515] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.497433 30262 leader_election.cc:290] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.497826 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c74d3ca66f42ad83ef54d9459b99fd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:41.498317 28346 leader_election.cc:343] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.499135 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet e6de1293572f4e75af3417f281b9a142 (table default.Table_f75651a1e55f4271b39388713dcd74aa [id=6705bb69561d435496d424c0d64f9879]) successfully deleted
I20260618 13:59:41.499537 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 142) took 6502 ms. Trace:
I20260618 13:59:41.499646 29746 rpcz_store.cc:276] 0618 13:59:34.997394 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.997716 (+ 322us) service_pool.cc:224] Handling call
0618 13:59:41.298733 (+6301017us) ts_tablet_manager.cc:1151] Deleting tablet 864f2602840c40fc8606b3784e671cba
0618 13:59:41.298744 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.304606 (+ 5862us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.422695 (+118089us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.499492 (+ 76797us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6242334}
I20260618 13:59:41.501778 28798 tablet_bootstrap.cc:492] T 80e99a85e83c46afa367e59e5ddb7dc7 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:41.502013 28798 ts_tablet_manager.cc:1403] T 80e99a85e83c46afa367e59e5ddb7dc7 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.127s user 0.010s sys 0.010s
I20260618 13:59:41.504223 29746 tablet_replica.cc:333] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:41.504463 29746 raft_consensus.cc:2243] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:41.504801 29746 raft_consensus.cc:2272] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.505930 30618 ts_tablet_manager.cc:1929] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:41.506583 29019 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 117) took 5882 ms. Trace:
I20260618 13:59:41.507161 29019 rpcz_store.cc:276] 0618 13:59:35.624565 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.449699 (+5825134us) service_pool.cc:224] Handling call
0618 13:59:41.456454 (+ 6755us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:41.456964 (+ 510us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:41.506302 (+ 49338us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:41.506412 (+ 110us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:41.506522 (+ 110us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.457494 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.457600 (+ 106us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.457605 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:41.457607 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:41.457689 (+ 82us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.457700 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.457705 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:41.457776 (+ 71us) tablet.cc:701] Row locks acquired
0618 13:59:41.457778 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:41.457824 (+ 46us) write_op.cc:270] Start()
0618 13:59:41.457885 (+ 61us) write_op.cc:276] Timestamp: P: 1781791181457796 usec, L: 0
0618 13:59:41.457889 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.458061 (+ 172us) log.cc:844] Serialized 433 byte log entry
0618 13:59:41.504097 (+ 46036us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.504399 (+ 302us) write_op.cc:301] APPLY: starting
0618 13:59:41.504417 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.504465 (+ 48us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.504498 (+ 33us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.504605 (+ 107us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.504639 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:41.504645 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:41.504725 (+ 80us) log.cc:844] Serialized 29 byte log entry
0618 13:59:41.504795 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.504816 (+ 21us) write_op.cc:454] Released schema lock
0618 13:59:41.504824 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:41.504915 (+ 91us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.504919 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":246,"apply.run_cpu_time_us":507,"apply.run_wall_time_us":508,"num_ops":2,"prepare.queue_time_us":48,"prepare.run_cpu_time_us":661,"prepare.run_wall_time_us":662,"raft.queue_time_us":95,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":46168}]]}
I20260618 13:59:41.509258 29019 master_service.cc:960] ReplaceTablet: received request to replace tablet 8fea86ca38a741238a59053151087f69 from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:41.509941 30618 log.cc:1199] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/b0d3d363766546c4a97b0763bad11f71
I20260618 13:59:41.511216 28798 raft_consensus.cc:359] T 80e99a85e83c46afa367e59e5ddb7dc7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.511409 28798 raft_consensus.cc:385] T 80e99a85e83c46afa367e59e5ddb7dc7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.511484 28798 raft_consensus.cc:740] T 80e99a85e83c46afa367e59e5ddb7dc7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
W20260618 13:59:41.512634 28346 consensus_peers.cc:597] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:41.513006 29026 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 118) took 5885 ms. Trace:
I20260618 13:59:41.513173 29026 rpcz_store.cc:276] 0618 13:59:35.627483 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.460715 (+5833232us) service_pool.cc:224] Handling call
0618 13:59:41.461776 (+ 1061us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:41.462243 (+ 467us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:41.512763 (+ 50520us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:41.512889 (+ 126us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:41.512972 (+ 83us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.462753 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.462861 (+ 108us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.462867 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:41.462869 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:41.462943 (+ 74us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.462953 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.462957 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:41.463023 (+ 66us) tablet.cc:701] Row locks acquired
0618 13:59:41.463026 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:41.463054 (+ 28us) write_op.cc:270] Start()
0618 13:59:41.463081 (+ 27us) write_op.cc:276] Timestamp: P: 1781791181463029 usec, L: 0
0618 13:59:41.463085 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.463234 (+ 149us) log.cc:844] Serialized 433 byte log entry
0618 13:59:41.504164 (+ 40930us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.504980 (+ 816us) write_op.cc:301] APPLY: starting
0618 13:59:41.504991 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.505024 (+ 33us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.505051 (+ 27us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.505141 (+ 90us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.505174 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:41.505180 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:41.505239 (+ 59us) log.cc:844] Serialized 29 byte log entry
0618 13:59:41.505281 (+ 42us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.505294 (+ 13us) write_op.cc:454] Released schema lock
0618 13:59:41.505300 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:41.505363 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.505367 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":788,"apply.run_cpu_time_us":373,"apply.run_wall_time_us":372,"num_ops":2,"prepare.queue_time_us":43,"prepare.run_cpu_time_us":562,"prepare.run_wall_time_us":562,"raft.queue_time_us":55,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":41063}]]}
I20260618 13:59:41.515033 30516 tablet.cc:1724] T 538077e459a44ed78c95f2e881b6e0b0 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:41.515379 29026 master_service.cc:960] ReplaceTablet: received request to replace tablet 5c75278db2234b11bf95f1cf5cc76edd from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:41.515770 29026 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 124) took 4923 ms. Trace:
I20260618 13:59:41.515935 29026 rpcz_store.cc:276] 0618 13:59:36.592513 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.515320 (+4922807us) service_pool.cc:224] Handling call
0618 13:59:41.515724 (+ 404us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260618 13:59:41.516765 29026 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_83d021ed148243ffb453cb5e2484b7b4" } modify_external_catalogs: true
I20260618 13:59:41.517019 29026 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_83d021ed148243ffb453cb5e2484b7b4" } modify_external_catalogs: true
I20260618 13:59:41.517501 29746 ts_tablet_manager.cc:1916] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.517791 30618 ts_tablet_manager.cc:1950] T b0d3d363766546c4a97b0763bad11f71 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:41.517735 29017 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 119) took 5888 ms. Trace:
I20260618 13:59:41.518043 29017 rpcz_store.cc:276] 0618 13:59:35.628912 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.465291 (+5836379us) service_pool.cc:224] Handling call
0618 13:59:41.466065 (+ 774us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:41.466522 (+ 457us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:41.517447 (+ 50925us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:41.517538 (+ 91us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:41.517689 (+ 151us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.466981 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.467095 (+ 114us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.467103 (+ 8us) write_op.cc:435] Acquired schema lock
0618 13:59:41.467107 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:41.467182 (+ 75us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.467191 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.467195 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:41.467273 (+ 78us) tablet.cc:701] Row locks acquired
0618 13:59:41.467275 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:41.467309 (+ 34us) write_op.cc:270] Start()
0618 13:59:41.467337 (+ 28us) write_op.cc:276] Timestamp: P: 1781791181467283 usec, L: 0
0618 13:59:41.467340 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.467511 (+ 171us) log.cc:844] Serialized 433 byte log entry
0618 13:59:41.504211 (+ 36700us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.505425 (+ 1214us) write_op.cc:301] APPLY: starting
0618 13:59:41.505435 (+ 10us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.505461 (+ 26us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.505490 (+ 29us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.505565 (+ 75us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.505590 (+ 25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:41.505596 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:41.505649 (+ 53us) log.cc:844] Serialized 29 byte log entry
0618 13:59:41.505695 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.505710 (+ 15us) write_op.cc:454] Released schema lock
0618 13:59:41.505717 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1182,"apply.run_cpu_time_us":350,"apply.run_wall_time_us":351,"num_ops":2,"prepare.queue_time_us":49,"prepare.run_cpu_time_us":634,"prepare.run_wall_time_us":719,"raft.queue_time_us":37,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":36853,"thread_start_us":6377,"threads_started":1}]]}
I20260618 13:59:41.498759 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c74d3ca66f42ad83ef54d9459b99fd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.519340 29017 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_1af4cab3531c47b5a9cd4fee3f8251c2" } modify_external_catalogs: true
I20260618 13:59:41.519009 28798 consensus_queue.cc:260] T 80e99a85e83c46afa367e59e5ddb7dc7 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:41.519484 28346 leader_election.cc:343] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.519507 29017 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_1af4cab3531c47b5a9cd4fee3f8251c2" } modify_external_catalogs: true
I20260618 13:59:41.519590 28346 leader_election.cc:304] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.499078 30584 raft_consensus.cc:515] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.520238 29024 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 122) took 5736 ms. Trace:
I20260618 13:59:41.520565 30584 leader_election.cc:290] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 election: Requested vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:41.493634 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 111) took 9343 ms. Trace:
I20260618 13:59:41.520752 29566 rpcz_store.cc:276] 0618 13:59:32.150487 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.150535 (+ 48us) service_pool.cc:224] Handling call
0618 13:59:41.459999 (+9309464us) ts_tablet_manager.cc:1151] Deleting tablet 04dac931e62a448d99633878eb70b92f
0618 13:59:41.460007 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.463833 (+ 3826us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.487684 (+ 23851us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.493615 (+ 5931us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9309307}
I20260618 13:59:41.520372 29024 rpcz_store.cc:276] 0618 13:59:35.783868 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.476735 (+5692867us) service_pool.cc:224] Handling call
0618 13:59:41.477615 (+ 880us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:41.478019 (+ 404us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:41.519990 (+ 41971us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:41.520075 (+ 85us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:41.520183 (+ 108us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.478458 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.478543 (+ 85us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.478551 (+ 8us) write_op.cc:435] Acquired schema lock
0618 13:59:41.478554 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:41.478626 (+ 72us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.478635 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.478640 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:41.478706 (+ 66us) tablet.cc:701] Row locks acquired
0618 13:59:41.478710 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:41.478742 (+ 32us) write_op.cc:270] Start()
0618 13:59:41.478768 (+ 26us) write_op.cc:276] Timestamp: P: 1781791181478715 usec, L: 0
0618 13:59:41.478772 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.478919 (+ 147us) log.cc:844] Serialized 433 byte log entry
0618 13:59:41.510898 (+ 31979us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.511535 (+ 637us) write_op.cc:301] APPLY: starting
0618 13:59:41.511548 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.511577 (+ 29us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.511610 (+ 33us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.511695 (+ 85us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.511721 (+ 26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:41.511727 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:41.511775 (+ 48us) log.cc:844] Serialized 29 byte log entry
0618 13:59:41.511813 (+ 38us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.511827 (+ 14us) write_op.cc:454] Released schema lock
0618 13:59:41.511832 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:41.511910 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.511914 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":598,"apply.run_cpu_time_us":365,"apply.run_wall_time_us":366,"num_ops":2,"prepare.queue_time_us":44,"prepare.run_cpu_time_us":545,"prepare.run_wall_time_us":597,"raft.queue_time_us":35,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":5,"replication_time_us":32099}]]}
I20260618 13:59:41.521057 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea4bc13eabc4dd98b4f96723498014c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:41.521224 28546 raft_consensus.cc:3060] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Advancing to term 2
I20260618 13:59:41.521992 29024 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_539ac216b6f4478eb994218d7d7980a4" } modify_external_catalogs: true
I20260618 13:59:41.522205 29024 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_539ac216b6f4478eb994218d7d7980a4" } modify_external_catalogs: true
I20260618 13:59:41.522440 30623 raft_consensus.cc:2749] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:41.522699 29020 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 121) took 5762 ms. Trace:
I20260618 13:59:41.522783 29020 rpcz_store.cc:276] 0618 13:59:35.760315 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.473375 (+5713060us) service_pool.cc:224] Handling call
0618 13:59:41.474331 (+ 956us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:41.474836 (+ 505us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:41.522491 (+ 47655us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:41.522575 (+ 84us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:41.522645 (+ 70us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.475313 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.475405 (+ 92us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.475410 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:41.475412 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:41.475492 (+ 80us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.475501 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.475505 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:41.475570 (+ 65us) tablet.cc:701] Row locks acquired
0618 13:59:41.475572 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:41.475602 (+ 30us) write_op.cc:270] Start()
0618 13:59:41.475634 (+ 32us) write_op.cc:276] Timestamp: P: 1781791181475574 usec, L: 0
0618 13:59:41.475637 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.476137 (+ 500us) log.cc:844] Serialized 433 byte log entry
0618 13:59:41.510733 (+ 34596us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.511036 (+ 303us) write_op.cc:301] APPLY: starting
0618 13:59:41.511048 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.511080 (+ 32us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.511100 (+ 20us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.511198 (+ 98us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.511228 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:41.511239 (+ 11us) write_op.cc:312] APPLY: finished
0618 13:59:41.511296 (+ 57us) log.cc:844] Serialized 29 byte log entry
0618 13:59:41.511347 (+ 51us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.511362 (+ 15us) write_op.cc:454] Released schema lock
0618 13:59:41.511380 (+ 18us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:41.511451 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.511455 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":260,"apply.run_cpu_time_us":403,"apply.run_wall_time_us":403,"num_ops":2,"prepare.queue_time_us":42,"prepare.run_cpu_time_us":595,"prepare.run_wall_time_us":966,"raft.queue_time_us":32,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":35068}]]}
I20260618 13:59:41.523589 28798 ts_tablet_manager.cc:1434] T 80e99a85e83c46afa367e59e5ddb7dc7 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.021s user 0.002s sys 0.000s
I20260618 13:59:41.523821 28546 raft_consensus.cc:2468] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 2.
I20260618 13:59:41.523911 28798 tablet_bootstrap.cc:492] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:41.524003 29020 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_9a10a9aa97574e7c89586d1e4e863171" } new_table_name: "default.Table_84be6926f4b64876b343577b85db3243" modify_external_catalogs: true
W20260618 13:59:41.521807 28617 leader_election.cc:343] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: cea4bc13eabc4dd98b4f96723498014c
I20260618 13:59:41.524237 28618 leader_election.cc:304] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:41.524618 30584 raft_consensus.cc:2804] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [term 2 FOLLOWER]: Leader election won for term 2
I20260618 13:59:41.524832 30584 raft_consensus.cc:697] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [term 2 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:41.525082 30584 consensus_queue.cc:237] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.525221 29566 tablet_replica.cc:333] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:41.525394 29566 raft_consensus.cc:2243] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [term 2 LEADER]: Raft consensus shutting down.
I20260618 13:59:41.525579 29566 pending_rounds.cc:70] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257: Trying to abort 1 pending ops.
I20260618 13:59:41.525620 29566 pending_rounds.cc:77] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7298216679259746304 op_type: NO_OP noop_request { }
I20260618 13:59:41.525701 29566 raft_consensus.cc:2889] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:41.525780 29566 raft_consensus.cc:2272] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.526116 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]) successfully deleted
I20260618 13:59:41.526785 29566 ts_tablet_manager.cc:1916] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.527630 28797 tablet_bootstrap.cc:492] T d0bcff62290c43c9be7cc496fbff6142 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:41.530652 28797 ts_tablet_manager.cc:1403] T d0bcff62290c43c9be7cc496fbff6142 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.055s user 0.002s sys 0.002s
I20260618 13:59:41.531673 28797 raft_consensus.cc:359] T d0bcff62290c43c9be7cc496fbff6142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.531949 28797 raft_consensus.cc:385] T d0bcff62290c43c9be7cc496fbff6142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.532132 28797 raft_consensus.cc:740] T d0bcff62290c43c9be7cc496fbff6142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.532413 28797 consensus_queue.cc:260] T d0bcff62290c43c9be7cc496fbff6142 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.533219 28797 ts_tablet_manager.cc:1434] T d0bcff62290c43c9be7cc496fbff6142 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:41.533756 28797 tablet_bootstrap.cc:492] T 3f5d4296952e48a48756e6f27140da87 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:41.535601 28797 tablet_bootstrap.cc:654] T 3f5d4296952e48a48756e6f27140da87 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.540371 28546 raft_consensus.cc:3060] T 9fa1a27fc9304e18b96cad9f923779f2 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.544754 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 04dac931e62a448d99633878eb70b92f (table default.Table_f37d21a9a59249d6a0b87c3788ff4873 [id=22eb9037fc66472793e9fd3f03fd7218]) successfully deleted
I20260618 13:59:41.548199 29566 ts_tablet_manager.cc:1929] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260618 13:59:41.551870 29566 log.cc:1199] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/be59f988f0e34b9480f859cf88b9c223
I20260618 13:59:41.559360 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 864f2602840c40fc8606b3784e671cba (table default.Table_fe8f6240599e467a92887643cc9d210b [id=3ac9a6b120d44ac2bae74d85d7bc6930]) successfully deleted
I20260618 13:59:41.560868 28798 tablet_bootstrap.cc:654] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.563476 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet 591465e2b57247a2b224cc8b09225811 with delete_type TABLET_DATA_DELETED (replaced by tablet d4667fe48f0f4eff9c6c8baabdab58d4) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.566092 30516 tablet.cc:1724] T ab100ebbe61a4321b9e76c730e5d46ed P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:41.565505 30513 tablet.cc:1724] T ab100ebbe61a4321b9e76c730e5d46ed P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:41.576327 30633 tablet.cc:1724] T ab100ebbe61a4321b9e76c730e5d46ed P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:41.574160 29566 ts_tablet_manager.cc:1950] T be59f988f0e34b9480f859cf88b9c223 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:41.577479 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 112) took 9425 ms. Trace:
I20260618 13:59:41.577685 29566 rpcz_store.cc:276] 0618 13:59:32.152332 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.152515 (+ 183us) service_pool.cc:224] Handling call
0618 13:59:41.525177 (+9372662us) ts_tablet_manager.cc:1151] Deleting tablet be59f988f0e34b9480f859cf88b9c223
0618 13:59:41.525187 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.544250 (+ 19063us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.548145 (+ 3895us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.577461 (+ 29316us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9372432}
I20260618 13:59:41.578104 29566 tablet_replica.cc:333] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:41.578338 29566 raft_consensus.cc:2243] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:41.578871 28537 raft_consensus.cc:3060] T 7766014ee9814f01ae7ac3e849ca3d50 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.580416 29566 raft_consensus.cc:2272] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.581370 29566 ts_tablet_manager.cc:1916] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.584273 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet be59f988f0e34b9480f859cf88b9c223 (table default.Table_b8741a7fd29f4e78a948b1c419595a85 [id=6f4d908593ae40f09e2c94cef218e1fd]) successfully deleted
I20260618 13:59:41.586723 28799 tablet_bootstrap.cc:492] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:41.586930 28799 ts_tablet_manager.cc:1403] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.187s user 0.005s sys 0.000s
I20260618 13:59:41.587980 28799 raft_consensus.cc:359] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.588304 28799 raft_consensus.cc:385] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.588410 28799 raft_consensus.cc:740] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.588685 28799 consensus_queue.cc:260] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.589982 28799 ts_tablet_manager.cc:1434] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:41.590946 28799 tablet_bootstrap.cc:492] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:41.592633 28799 tablet_bootstrap.cc:654] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.595542 29566 ts_tablet_manager.cc:1929] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:41.602658 29566 log.cc:1199] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/c3d523419ff244f085d6eac1a3231973
I20260618 13:59:41.603152 29566 ts_tablet_manager.cc:1950] T c3d523419ff244f085d6eac1a3231973 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:41.604609 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 115) took 9446 ms. Trace:
I20260618 13:59:41.604789 29566 rpcz_store.cc:276] 0618 13:59:32.158030 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.158082 (+ 52us) service_pool.cc:224] Handling call
0618 13:59:41.578062 (+9419980us) ts_tablet_manager.cc:1151] Deleting tablet c3d523419ff244f085d6eac1a3231973
0618 13:59:41.578071 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.592064 (+ 13993us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.595475 (+ 3411us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.604586 (+ 9111us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9419826}
I20260618 13:59:41.605587 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet c3d523419ff244f085d6eac1a3231973 (table default.Table_176a739c87ee45798e2470f8ec00d632 [id=2534d29e39c14366b10492a7c6a92947]) successfully deleted
I20260618 13:59:41.605998 29566 tablet_replica.cc:333] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:41.606328 29566 raft_consensus.cc:2243] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:41.607606 29566 raft_consensus.cc:2272] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.608178 29566 ts_tablet_manager.cc:1916] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:41.611430 29025 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.2:52853 (request call id 123) took 1672 ms (client timeout 1000 ms). Trace:
W20260618 13:59:41.611616 29025 rpcz_store.cc:269] 0618 13:59:39.938761 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.243398 (+304637us) service_pool.cc:224] Handling call
0618 13:59:41.611386 (+1367988us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5110,"reported_tablets":190}
I20260618 13:59:41.614295 29025 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_c70d6c5b0dc94b33b4c5772edd2b9dca" } new_table_name: "default.Table_93fe53cac2f54d73b1e66cf1661ca928" modify_external_catalogs: true
I20260618 13:59:41.627946 29566 ts_tablet_manager.cc:1929] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:41.628135 29566 log.cc:1199] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/015494374645469785f1f978eaf0e9a5
I20260618 13:59:41.628594 29566 ts_tablet_manager.cc:1950] T 015494374645469785f1f978eaf0e9a5 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:41.629338 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 116) took 9470 ms. Trace:
I20260618 13:59:41.629498 29566 rpcz_store.cc:276] 0618 13:59:32.158818 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.158861 (+ 43us) service_pool.cc:224] Handling call
0618 13:59:41.605937 (+9447076us) ts_tablet_manager.cc:1151] Deleting tablet 015494374645469785f1f978eaf0e9a5
0618 13:59:41.605949 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.624991 (+ 19042us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.627899 (+ 2908us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.629319 (+ 1420us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9446878}
I20260618 13:59:41.630003 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 015494374645469785f1f978eaf0e9a5 (table default.Table_7699960fdee24242bf51bcfcbd6f68c2 [id=3b9ab803536344d3a59ff77b2b58ae31]) successfully deleted
I20260618 13:59:41.630264 29566 tablet_replica.cc:333] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:41.630599 29566 raft_consensus.cc:2243] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:41.632695 29566 raft_consensus.cc:2272] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.633226 29566 ts_tablet_manager.cc:1916] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.628861 30262 raft_consensus.cc:493] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.641999 30262 raft_consensus.cc:515] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.642664 30262 leader_election.cc:290] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.639097 30622 raft_consensus.cc:493] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.642975 30622 raft_consensus.cc:515] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.643085 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89b26d1595642ff82381f8baddfb99a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.643639 30622 leader_election.cc:290] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.644035 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89b26d1595642ff82381f8baddfb99a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:41.644127 28346 leader_election.cc:343] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.644522 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779755a607bf46219900412b527e58b5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.645301 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779755a607bf46219900412b527e58b5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.645941 28797 tablet_bootstrap.cc:492] T 3f5d4296952e48a48756e6f27140da87 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
W20260618 13:59:41.645970 28346 leader_election.cc:343] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.646064 28797 ts_tablet_manager.cc:1403] T 3f5d4296952e48a48756e6f27140da87 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.113s user 0.005s sys 0.001s
W20260618 13:59:41.646114 28346 leader_election.cc:343] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.646176 28346 leader_election.cc:304] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.646701 28797 raft_consensus.cc:359] T 3f5d4296952e48a48756e6f27140da87 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.646572 30622 raft_consensus.cc:2749] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:41.646871 28797 raft_consensus.cc:385] T 3f5d4296952e48a48756e6f27140da87 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.646917 28797 raft_consensus.cc:740] T 3f5d4296952e48a48756e6f27140da87 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.647094 28797 consensus_queue.cc:260] T 3f5d4296952e48a48756e6f27140da87 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:41.647161 28346 leader_election.cc:343] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.647256 28346 leader_election.cc:304] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.647509 28797 ts_tablet_manager.cc:1434] T 3f5d4296952e48a48756e6f27140da87 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260618 13:59:41.647697 30622 raft_consensus.cc:2749] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:41.647764 28797 tablet_bootstrap.cc:492] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:41.652082 29746 ts_tablet_manager.cc:1929] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:41.652230 29746 log.cc:1199] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/75b9c0754beb4e0e8ed3a7c1e9f368a9
I20260618 13:59:41.652691 29746 ts_tablet_manager.cc:1950] T 75b9c0754beb4e0e8ed3a7c1e9f368a9 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
W20260618 13:59:41.654978 28990 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3a783590159f4a80964fbbcb4a24eef9 consensus state has no leader
I20260618 13:59:41.658532 29566 ts_tablet_manager.cc:1929] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:41.658710 29566 log.cc:1199] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/ac58e939730c4af681c2f08bcfbb6602
I20260618 13:59:41.659118 29566 ts_tablet_manager.cc:1950] T ac58e939730c4af681c2f08bcfbb6602 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:41.659401 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 143) took 6661 ms. Trace:
I20260618 13:59:41.659518 29746 rpcz_store.cc:276] 0618 13:59:34.997489 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.997740 (+ 251us) service_pool.cc:224] Handling call
0618 13:59:41.504157 (+6506417us) ts_tablet_manager.cc:1151] Deleting tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9
0618 13:59:41.504167 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.555103 (+ 50936us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.651990 (+ 96887us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.659353 (+ 7363us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6447705}
I20260618 13:59:41.659888 29746 tablet_replica.cc:333] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:41.660116 29746 raft_consensus.cc:2243] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:41.660315 28798 tablet_bootstrap.cc:492] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:41.660319 29746 raft_consensus.cc:2272] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.660476 28798 ts_tablet_manager.cc:1403] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.137s user 0.003s sys 0.000s
I20260618 13:59:41.660601 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 117) took 9500 ms. Trace:
I20260618 13:59:41.660904 29746 ts_tablet_manager.cc:1916] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.661011 29566 rpcz_store.cc:276] 0618 13:59:32.159664 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.159708 (+ 44us) service_pool.cc:224] Handling call
0618 13:59:41.630220 (+9470512us) ts_tablet_manager.cc:1151] Deleting tablet ac58e939730c4af681c2f08bcfbb6602
0618 13:59:41.630230 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.654930 (+ 24700us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.658484 (+ 3554us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.660578 (+ 2094us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9470368}
I20260618 13:59:41.661375 29566 tablet_replica.cc:333] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:41.661556 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet ac58e939730c4af681c2f08bcfbb6602 (table default.Table_ceb33d60d86e4fd898ae67c78da94a55 [id=3251ae257e0f49b99352dcc1ef004588]) successfully deleted
I20260618 13:59:41.664419 29566 raft_consensus.cc:2243] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:41.664714 29566 raft_consensus.cc:2272] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.664640 28798 raft_consensus.cc:359] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.664832 28798 raft_consensus.cc:385] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.664925 28798 raft_consensus.cc:740] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.665318 29566 ts_tablet_manager.cc:1916] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.666970 28798 consensus_queue.cc:260] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.667505 28798 ts_tablet_manager.cc:1434] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260618 13:59:41.669368 28798 tablet_bootstrap.cc:492] T 340f6bbf18eb412ca3afec237ff7752d P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:41.671664 28798 tablet_bootstrap.cc:654] T 340f6bbf18eb412ca3afec237ff7752d P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.673243 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:41.673556 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:41.674067 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:41.674645 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:41.674834 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:41.675248 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:41.676497 29566 ts_tablet_manager.cc:1929] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:41.681619 28797 tablet_bootstrap.cc:654] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.683275 30584 raft_consensus.cc:493] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.688439 30584 raft_consensus.cc:515] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.689040 30584 leader_election.cc:290] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749)
I20260618 13:59:41.690161 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4667fe48f0f4eff9c6c8baabdab58d4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.690289 30551 raft_consensus.cc:493] T 80e99a85e83c46afa367e59e5ddb7dc7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.690491 30551 raft_consensus.cc:515] T 80e99a85e83c46afa367e59e5ddb7dc7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:41.690770 28618 leader_election.cc:343] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.691242 30551 leader_election.cc:290] T 80e99a85e83c46afa367e59e5ddb7dc7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:41.692046 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e99a85e83c46afa367e59e5ddb7dc7" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
W20260618 13:59:41.692759 28617 leader_election.cc:343] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: d4667fe48f0f4eff9c6c8baabdab58d4
I20260618 13:59:41.692875 28617 leader_election.cc:304] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:41.693264 30584 raft_consensus.cc:2749] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:41.694262 28799 tablet_bootstrap.cc:492] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:41.694478 28799 ts_tablet_manager.cc:1403] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.104s user 0.003s sys 0.000s
I20260618 13:59:41.695021 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80e99a85e83c46afa367e59e5ddb7dc7" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.695358 28799 raft_consensus.cc:359] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.695519 28799 raft_consensus.cc:385] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.695576 28799 raft_consensus.cc:740] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.695720 28484 leader_election.cc:304] T 80e99a85e83c46afa367e59e5ddb7dc7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.695799 28799 consensus_queue.cc:260] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.696197 30551 raft_consensus.cc:2749] T 80e99a85e83c46afa367e59e5ddb7dc7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:41.701056 29566 log.cc:1199] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/7572d17724d249739be1a1cb9468f909
I20260618 13:59:41.701799 29566 ts_tablet_manager.cc:1950] T 7572d17724d249739be1a1cb9468f909 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:41.703189 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 118) took 9542 ms. Trace:
I20260618 13:59:41.704123 29566 rpcz_store.cc:276] 0618 13:59:32.161208 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.161264 (+ 56us) service_pool.cc:224] Handling call
0618 13:59:41.661334 (+9500070us) ts_tablet_manager.cc:1151] Deleting tablet 7572d17724d249739be1a1cb9468f909
0618 13:59:41.661343 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.668766 (+ 7423us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.676445 (+ 7679us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.703171 (+ 26726us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9499819}
I20260618 13:59:41.704761 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 7572d17724d249739be1a1cb9468f909 (table default.Table_b583e9f1f2464ebe9df9b4663bb0a4f8 [id=67643f46e18742859d79b0608aeaad23]) successfully deleted
I20260618 13:59:41.704861 29566 tablet_replica.cc:333] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:41.705057 29566 raft_consensus.cc:2243] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:41.705435 29566 raft_consensus.cc:2272] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.705863 28799 ts_tablet_manager.cc:1434] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260618 13:59:41.706099 29566 ts_tablet_manager.cc:1916] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.711207 28797 tablet_bootstrap.cc:492] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:41.711341 28797 ts_tablet_manager.cc:1403] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.064s user 0.002s sys 0.008s
I20260618 13:59:41.712026 28797 raft_consensus.cc:359] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.712188 28797 raft_consensus.cc:385] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.712241 28797 raft_consensus.cc:740] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.712476 28797 consensus_queue.cc:260] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.715988 29746 ts_tablet_manager.cc:1929] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:41.716115 29746 log.cc:1199] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/827cc0987bd541759e334fb620f04ed5
I20260618 13:59:41.716513 29746 ts_tablet_manager.cc:1950] T 827cc0987bd541759e334fb620f04ed5 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:41.716838 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 140) took 6719 ms. Trace:
I20260618 13:59:41.716939 29746 rpcz_store.cc:276] 0618 13:59:34.997194 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.997668 (+ 474us) service_pool.cc:224] Handling call
0618 13:59:41.659813 (+6662145us) ts_tablet_manager.cc:1151] Deleting tablet 827cc0987bd541759e334fb620f04ed5
0618 13:59:41.659824 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.680495 (+ 20671us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.715912 (+ 35417us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.716805 (+ 893us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6603323}
I20260618 13:59:41.717463 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 827cc0987bd541759e334fb620f04ed5 (table default.Table_9bfb035b965742abaaf21413be078bf6 [id=9be95b9adc544c0e9675071330a21225]) successfully deleted
I20260618 13:59:41.719167 28799 tablet_bootstrap.cc:492] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:41.719348 28797 ts_tablet_manager.cc:1434] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260618 13:59:41.720458 28797 tablet_bootstrap.cc:492] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:41.721101 28799 tablet_bootstrap.cc:654] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.722347 28797 tablet_bootstrap.cc:654] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.725023 29746 tablet_replica.cc:333] T 7d4144b1b5ea4e968672616bc22bead8 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:41.725294 29746 raft_consensus.cc:2243] T 7d4144b1b5ea4e968672616bc22bead8 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:41.725550 29746 raft_consensus.cc:2272] T 7d4144b1b5ea4e968672616bc22bead8 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.726100 29746 ts_tablet_manager.cc:1916] T 7d4144b1b5ea4e968672616bc22bead8 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.730798 30584 consensus_queue.cc:1048] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:41.731618 30622 raft_consensus.cc:493] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.731741 30622 raft_consensus.cc:515] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.732290 30622 leader_election.cc:290] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.733295 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bcff62290c43c9be7cc496fbff6142" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.734593 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:41.734869 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:41.734997 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_01e1e5741bb54f78996857f6b79fc5d7 [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 2 got 0
I20260618 13:59:41.735623 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:41.736050 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 2128dbec33db453ba51070767a15b21e (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]). Expected version 2 got 0
I20260618 13:59:41.736334 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:41.736687 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:41.737419 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet ba53ad1988064049a61fcca3a85f8779 (table default.Table_b7f097c1abe1410a800ce5aa25771740 [id=6c30b030a60b47e2be48dfe6e9cfdfdb]). Expected version 1 got 0
I20260618 13:59:41.738133 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:41.743211 28798 tablet_bootstrap.cc:492] T 340f6bbf18eb412ca3afec237ff7752d P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:41.743357 28798 ts_tablet_manager.cc:1403] T 340f6bbf18eb412ca3afec237ff7752d P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.074s user 0.000s sys 0.003s
I20260618 13:59:41.744074 28798 raft_consensus.cc:359] T 340f6bbf18eb412ca3afec237ff7752d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.744277 28798 raft_consensus.cc:385] T 340f6bbf18eb412ca3afec237ff7752d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.744350 28798 raft_consensus.cc:740] T 340f6bbf18eb412ca3afec237ff7752d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.744583 28798 consensus_queue.cc:260] T 340f6bbf18eb412ca3afec237ff7752d P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.744791 28797 tablet_bootstrap.cc:492] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:41.744920 28797 ts_tablet_manager.cc:1403] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.000s
I20260618 13:59:41.745924 28797 raft_consensus.cc:359] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.745941 30513 tablet.cc:1724] T ba53ad1988064049a61fcca3a85f8779 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:41.746110 28797 raft_consensus.cc:385] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.746168 28797 raft_consensus.cc:740] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.746371 28797 consensus_queue.cc:260] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.746948 28797 ts_tablet_manager.cc:1434] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:41.747232 28797 tablet_bootstrap.cc:492] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:41.748459 28798 ts_tablet_manager.cc:1434] T 340f6bbf18eb412ca3afec237ff7752d P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
I20260618 13:59:41.748724 28798 tablet_bootstrap.cc:492] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:41.750272 28798 tablet_bootstrap.cc:654] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.751591 30633 tablet.cc:1724] T ba53ad1988064049a61fcca3a85f8779 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260618 13:59:41.756726 28618 consensus_peers.cc:597] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:41.761822 28799 tablet_bootstrap.cc:492] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:41.761955 28799 ts_tablet_manager.cc:1403] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.055s user 0.002s sys 0.001s
I20260618 13:59:41.762643 28799 raft_consensus.cc:359] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.762813 28799 raft_consensus.cc:385] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.762866 28799 raft_consensus.cc:740] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.763090 28799 consensus_queue.cc:260] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.763572 28799 ts_tablet_manager.cc:1434] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:41.763847 28799 tablet_bootstrap.cc:492] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:41.765234 28799 tablet_bootstrap.cc:654] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.766650 28798 tablet_bootstrap.cc:492] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:41.766795 28798 ts_tablet_manager.cc:1403] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.002s
I20260618 13:59:41.767519 29746 ts_tablet_manager.cc:1929] T 7d4144b1b5ea4e968672616bc22bead8 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:41.767665 29746 log.cc:1199] T 7d4144b1b5ea4e968672616bc22bead8 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/7d4144b1b5ea4e968672616bc22bead8
I20260618 13:59:41.768079 29746 ts_tablet_manager.cc:1950] T 7d4144b1b5ea4e968672616bc22bead8 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:41.767587 28798 raft_consensus.cc:359] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.768487 28798 raft_consensus.cc:385] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.768589 28798 raft_consensus.cc:740] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.768862 28798 consensus_queue.cc:260] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.774909 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bcff62290c43c9be7cc496fbff6142" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.775097 28683 raft_consensus.cc:2468] T d0bcff62290c43c9be7cc496fbff6142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:41.775501 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 139) took 6771 ms. Trace:
I20260618 13:59:41.775672 29746 rpcz_store.cc:276] 0618 13:59:34.997042 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.997640 (+ 598us) service_pool.cc:224] Handling call
0618 13:59:41.724946 (+6727306us) ts_tablet_manager.cc:1151] Deleting tablet 7d4144b1b5ea4e968672616bc22bead8
0618 13:59:41.724958 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.752893 (+ 27935us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.767444 (+ 14551us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.768391 (+ 947us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6668423}
I20260618 13:59:41.776463 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 7d4144b1b5ea4e968672616bc22bead8 (table default.Table_6ad6cf4da29143ff9ad3187cfcb95f6e [id=c2704c9f28bf4351a809b58bf4af056f]) successfully deleted
I20260618 13:59:41.776840 29746 tablet_replica.cc:333] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:41.777097 29746 raft_consensus.cc:2243] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:41.777324 29746 raft_consensus.cc:2272] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.777881 29746 ts_tablet_manager.cc:1916] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.778188 28346 leader_election.cc:304] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:41.781349 28537 raft_consensus.cc:3060] T aa93f065b4d64e7b85e30f0690fae541 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.782384 30622 raft_consensus.cc:2804] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:41.782481 30622 raft_consensus.cc:493] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:41.782545 30622 raft_consensus.cc:3060] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.782567 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c9ceb3af08fa4618b81cf6f89d7bd0df (table default.Table_21242eba3cb64c7c91082e296439cfe2 [id=10a4f5ef579746b68d8dd37d83f14c3e]). Expected version 1 got 0
I20260618 13:59:41.782959 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:41.783151 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:41.783850 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:41.784090 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:41.784226 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:41.785126 30622 raft_consensus.cc:515] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.785672 30622 leader_election.cc:290] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.787528 28797 tablet_bootstrap.cc:654] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.789502 29566 ts_tablet_manager.cc:1929] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:41.789597 29566 log.cc:1199] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/ab521028ac1a4d0da013112d76bb169f
I20260618 13:59:41.789978 29566 ts_tablet_manager.cc:1950] T ab521028ac1a4d0da013112d76bb169f P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:41.790731 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 119) took 9628 ms. Trace:
I20260618 13:59:41.790764 28798 ts_tablet_manager.cc:1434] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.024s user 0.000s sys 0.002s
I20260618 13:59:41.790808 29566 rpcz_store.cc:276] 0618 13:59:32.162458 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.162514 (+ 56us) service_pool.cc:224] Handling call
0618 13:59:41.704814 (+9542300us) ts_tablet_manager.cc:1151] Deleting tablet ab521028ac1a4d0da013112d76bb169f
0618 13:59:41.704823 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.709509 (+ 4686us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.789449 (+ 79940us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.790712 (+ 1263us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9542071}
I20260618 13:59:41.791116 28798 tablet_bootstrap.cc:492] T 054a7afb3b644aa49df78302cf8f02c9 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:41.791306 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet ab521028ac1a4d0da013112d76bb169f (table default.Table_3deacb2c9668491a85bbb09ae3f57d66 [id=431a660bfbed46248a603dc7781cf543]) successfully deleted
I20260618 13:59:41.791923 29566 tablet_replica.cc:333] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:41.792325 29566 raft_consensus.cc:2243] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:41.792533 29566 raft_consensus.cc:2272] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.792863 28798 tablet_bootstrap.cc:654] T 054a7afb3b644aa49df78302cf8f02c9 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.793208 29566 ts_tablet_manager.cc:1916] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.784394 29007 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:41.804282 28537 raft_consensus.cc:3060] T 438663f365e34fab87e34f434920cecf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:41.808679 28346 leader_election.cc:343] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.811296 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bcff62290c43c9be7cc496fbff6142" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:41.811518 28683 raft_consensus.cc:3060] T d0bcff62290c43c9be7cc496fbff6142 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.812085 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bcff62290c43c9be7cc496fbff6142" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:41.814033 28683 raft_consensus.cc:2468] T d0bcff62290c43c9be7cc496fbff6142 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:41.815596 28799 tablet_bootstrap.cc:492] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:41.815735 28799 ts_tablet_manager.cc:1403] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.052s user 0.002s sys 0.000s
I20260618 13:59:41.816394 28799 raft_consensus.cc:359] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.816576 28799 raft_consensus.cc:385] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.816640 28799 raft_consensus.cc:740] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.816859 28799 consensus_queue.cc:260] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.818238 28799 ts_tablet_manager.cc:1434] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:41.819187 28799 tablet_bootstrap.cc:492] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:41.822834 29566 ts_tablet_manager.cc:1929] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:41.823472 28346 leader_election.cc:304] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:41.823971 30622 raft_consensus.cc:2804] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:41.825173 30622 raft_consensus.cc:697] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:41.825481 30622 consensus_queue.cc:237] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:41.824090 29303 raft_consensus.cc:493] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.833811 29303 raft_consensus.cc:515] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.834672 29303 leader_election.cc:290] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:41.835219 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4539f767c054e9e8ebc1f3f56108e7e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:41.835399 28408 raft_consensus.cc:2468] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:41.835695 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4539f767c054e9e8ebc1f3f56108e7e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.833801 29566 log.cc:1199] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/5e25027f423144b088dad955c7282921
I20260618 13:59:41.836130 29566 ts_tablet_manager.cc:1950] T 5e25027f423144b088dad955c7282921 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
W20260618 13:59:41.836155 28618 leader_election.cc:343] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.836264 28618 leader_election.cc:304] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:41.836506 30572 raft_consensus.cc:2804] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:41.836576 30572 raft_consensus.cc:493] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:41.836632 30572 raft_consensus.cc:3060] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.836860 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 120) took 9670 ms. Trace:
I20260618 13:59:41.836934 29566 rpcz_store.cc:276] 0618 13:59:32.166592 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.166862 (+ 270us) service_pool.cc:224] Handling call
0618 13:59:41.791870 (+9625008us) ts_tablet_manager.cc:1151] Deleting tablet 5e25027f423144b088dad955c7282921
0618 13:59:41.791882 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.800263 (+ 8381us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.822767 (+ 22504us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.836841 (+ 14074us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9624798}
I20260618 13:59:41.838135 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 5e25027f423144b088dad955c7282921 (table default.Table_a7ff69763fdb4ff3b26b180bf05907d9 [id=e39e2c5718794a748361d7b453c65404]) successfully deleted
I20260618 13:59:41.838665 30262 consensus_queue.cc:1048] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:41.839154 30572 raft_consensus.cc:515] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:41.839381 28346 consensus_peers.cc:597] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:41.839530 28346 leader_election.cc:343] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.840049 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4539f767c054e9e8ebc1f3f56108e7e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:41.840199 30572 leader_election.cc:290] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:41.840485 28618 leader_election.cc:343] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.841604 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4539f767c054e9e8ebc1f3f56108e7e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:41.841748 28408 raft_consensus.cc:3060] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.844327 28408 raft_consensus.cc:2468] T d4539f767c054e9e8ebc1f3f56108e7e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:41.844704 28617 leader_election.cc:304] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:41.845086 29303 raft_consensus.cc:2804] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:41.845208 29303 raft_consensus.cc:697] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:41.845414 29303 consensus_queue.cc:237] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.846297 29019 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 8fea86ca38a741238a59053151087f69
I20260618 13:59:41.846930 29019 catalog_manager.cc:6784] ReplaceTablet: tablet 8fea86ca38a741238a59053151087f69 deleted and replaced by tablet 3f9ec5f31f6c4d37b7c3ec585ed3211b
I20260618 13:59:41.847754 29019 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 123) took 5721 ms. Trace:
I20260618 13:59:41.847884 29019 rpcz_store.cc:276] 0618 13:59:36.126509 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.509169 (+5382660us) service_pool.cc:224] Handling call
0618 13:59:41.512600 (+ 3431us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb469154b 00007fcbab0ed107 00007fcbab0ece35 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:41.847717 (+335117us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.836515 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.836634 (+ 119us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.836640 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:41.836642 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:41.836752 (+ 110us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.836765 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.836770 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:41.836861 (+ 91us) tablet.cc:701] Row locks acquired
0618 13:59:41.836863 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:41.836902 (+ 39us) write_op.cc:270] Start()
0618 13:59:41.836937 (+ 35us) write_op.cc:276] Timestamp: P: 1781791181836836 usec, L: 0
0618 13:59:41.836942 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.837110 (+ 168us) log.cc:844] Serialized 625 byte log entry
0618 13:59:41.841039 (+ 3929us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.841196 (+ 157us) write_op.cc:301] APPLY: starting
0618 13:59:41.841213 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.845301 (+ 4088us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.845349 (+ 48us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.845513 (+ 164us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.845595 (+ 82us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:41.845607 (+ 12us) write_op.cc:312] APPLY: finished
0618 13:59:41.845706 (+ 99us) log.cc:844] Serialized 29 byte log entry
0618 13:59:41.845783 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.845824 (+ 41us) write_op.cc:454] Released schema lock
0618 13:59:41.845834 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:41.845925 (+ 91us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.845931 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":6363008,"child_traces":[["op",{"apply.queue_time_us":110,"apply.run_cpu_time_us":697,"apply.run_wall_time_us":4709,"num_ops":2,"prepare.queue_time_us":140,"prepare.run_cpu_time_us":708,"prepare.run_wall_time_us":708,"raft.queue_time_us":165,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":254,"replication_time_us":4068}]]}
I20260618 13:59:41.848965 29019 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_25d0ebc0b8a4422087089d856a3e1f5d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:41.849674 29019 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_25d0ebc0b8a4422087089d856a3e1f5d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:41.852330 28644 tablet_service.cc:1558] Processing DeleteTablet for tablet 8fea86ca38a741238a59053151087f69 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f9ec5f31f6c4d37b7c3ec585ed3211b) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:41.852605 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 8fea86ca38a741238a59053151087f69 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f9ec5f31f6c4d37b7c3ec585ed3211b) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:41.852749 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet 8fea86ca38a741238a59053151087f69 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f9ec5f31f6c4d37b7c3ec585ed3211b) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:41.852847 30618 tablet_replica.cc:333] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:41.853024 30618 raft_consensus.cc:2243] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:41.853179 30618 raft_consensus.cc:2272] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.853727 30618 ts_tablet_manager.cc:1916] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.854401 30577 raft_consensus.cc:493] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.854555 30577 raft_consensus.cc:515] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.855093 30577 leader_election.cc:290] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:41.856278 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9f9649587b4c31951204b8bebe7ed8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:41.856446 28407 raft_consensus.cc:2468] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:41.856561 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9f9649587b4c31951204b8bebe7ed8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:41.856959 28618 leader_election.cc:343] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.857189 28617 leader_election.cc:304] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:41.857538 30577 raft_consensus.cc:2804] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:41.857611 30577 raft_consensus.cc:493] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:41.857662 30577 raft_consensus.cc:3060] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.859519 30618 ts_tablet_manager.cc:1929] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:41.859616 30618 log.cc:1199] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/8fea86ca38a741238a59053151087f69
I20260618 13:59:41.859954 30618 ts_tablet_manager.cc:1950] T 8fea86ca38a741238a59053151087f69 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:41.860072 30577 raft_consensus.cc:515] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.860612 30577 leader_election.cc:290] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:41.861300 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9f9649587b4c31951204b8bebe7ed8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:41.861589 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9f9649587b4c31951204b8bebe7ed8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:41.861737 28407 raft_consensus.cc:3060] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.863457 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 8fea86ca38a741238a59053151087f69 (table default.Table_7ae70108b1714e5ca374cc653bed7b33 [id=0cf9d7859b7d4d95bf769070c43003d3]) successfully deleted
I20260618 13:59:41.864136 29566 tablet_replica.cc:333] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:41.864137 28407 raft_consensus.cc:2468] T 5b9f9649587b4c31951204b8bebe7ed8 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:41.864365 29566 raft_consensus.cc:2243] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:41.864727 29566 raft_consensus.cc:2272] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:41.865060 28618 leader_election.cc:343] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.865254 29566 ts_tablet_manager.cc:1916] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.865659 28617 leader_election.cc:304] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:41.866088 30572 raft_consensus.cc:2804] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:41.866196 30572 raft_consensus.cc:697] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:41.866432 30551 raft_consensus.cc:493] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:41.866585 30551 raft_consensus.cc:515] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.866794 29019 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 130) took 3806 ms. Trace:
I20260618 13:59:41.867197 30551 leader_election.cc:290] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:41.867911 30572 consensus_queue.cc:237] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.866943 29019 rpcz_store.cc:276] 0618 13:59:38.060503 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.848716 (+3788213us) service_pool.cc:224] Handling call
0618 13:59:41.849788 (+ 1072us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:41.857548 (+ 7760us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:41.866528 (+ 8980us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:41.866654 (+ 126us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:41.866752 (+ 98us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.858173 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.858298 (+ 125us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.858303 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:41.858305 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:41.858390 (+ 85us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.858404 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.858408 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:41.858504 (+ 96us) tablet.cc:701] Row locks acquired
0618 13:59:41.858507 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:41.858546 (+ 39us) write_op.cc:270] Start()
0618 13:59:41.858580 (+ 34us) write_op.cc:276] Timestamp: P: 1781791181858475 usec, L: 0
0618 13:59:41.858584 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.858747 (+ 163us) log.cc:844] Serialized 433 byte log entry
0618 13:59:41.862479 (+ 3732us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.862648 (+ 169us) write_op.cc:301] APPLY: starting
0618 13:59:41.862668 (+ 20us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.862738 (+ 70us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.862771 (+ 33us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.862932 (+ 161us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.862979 (+ 47us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:41.862990 (+ 11us) write_op.cc:312] APPLY: finished
0618 13:59:41.863082 (+ 92us) log.cc:844] Serialized 29 byte log entry
0618 13:59:41.863145 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.863169 (+ 24us) write_op.cc:454] Released schema lock
0618 13:59:41.863176 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:41.863247 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.863253 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":116,"apply.run_cpu_time_us":589,"apply.run_wall_time_us":589,"num_ops":2,"prepare.queue_time_us":141,"prepare.run_cpu_time_us":697,"prepare.run_wall_time_us":697,"raft.queue_time_us":194,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":198,"replication_time_us":3863}]]}
W20260618 13:59:41.868146 28483 leader_election.cc:343] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: c7f42997416841a7b2c1ad347d2e9282
I20260618 13:59:41.869112 29019 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_35260f90f6814fa3915b4280cff96d1b" } new_table_name: "default.Table_2dfebd62a146446f999f1a7a1bd1af79" modify_external_catalogs: true
W20260618 13:59:41.870112 28484 leader_election.cc:343] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: c7f42997416841a7b2c1ad347d2e9282
I20260618 13:59:41.870210 28484 leader_election.cc:304] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.870548 30551 raft_consensus.cc:2749] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:41.876709 30262 raft_consensus.cc:493] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.876856 30262 raft_consensus.cc:515] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.877498 30262 leader_election.cc:290] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.878852 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "732c8801c319400283806aac7f15917e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:41.884784 28346 leader_election.cc:343] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.885159 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "732c8801c319400283806aac7f15917e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:41.885797 28346 leader_election.cc:343] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.885994 28346 leader_election.cc:304] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.886291 30623 raft_consensus.cc:2749] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:41.896479 29746 ts_tablet_manager.cc:1929] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:41.900360 29746 log.cc:1199] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/68b8293a7c924a449307ab88a1cf9290
I20260618 13:59:41.901168 29746 ts_tablet_manager.cc:1950] T 68b8293a7c924a449307ab88a1cf9290 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:41.903005 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 137) took 6906 ms. Trace:
I20260618 13:59:41.905177 29746 rpcz_store.cc:276] 0618 13:59:34.996808 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.997581 (+ 773us) service_pool.cc:224] Handling call
0618 13:59:41.776770 (+6779189us) ts_tablet_manager.cc:1151] Deleting tablet 68b8293a7c924a449307ab88a1cf9290
0618 13:59:41.776780 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.851205 (+ 74425us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.896392 (+ 45187us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.902980 (+ 6588us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6720212}
I20260618 13:59:41.907014 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 68b8293a7c924a449307ab88a1cf9290 (table default.Table_c97dcabd4c2b45a18f445a099ec7ace9 [id=38bde7a301814aa285e4bd30bcc527aa]) successfully deleted
I20260618 13:59:41.907648 29746 tablet_replica.cc:333] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:41.910712 29566 ts_tablet_manager.cc:1929] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:41.911962 29566 log.cc:1199] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/283523e5b22843ba935eabe6283d3a99
I20260618 13:59:41.912427 29566 ts_tablet_manager.cc:1950] T 283523e5b22843ba935eabe6283d3a99 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:41.915911 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 121) took 9745 ms. Trace:
I20260618 13:59:41.916102 29566 rpcz_store.cc:276] 0618 13:59:32.170308 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.170353 (+ 45us) service_pool.cc:224] Handling call
0618 13:59:41.864070 (+9693717us) ts_tablet_manager.cc:1151] Deleting tablet 283523e5b22843ba935eabe6283d3a99
0618 13:59:41.864082 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.898535 (+ 34453us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.910643 (+ 12108us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.915885 (+ 5242us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9580744}
I20260618 13:59:41.916841 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 283523e5b22843ba935eabe6283d3a99 (table default.Table_d16c53f18154437e8d06e88da66faf13 [id=cf2c2a972733485a9ba1d6f1e99937a1]) successfully deleted
I20260618 13:59:41.917361 29566 tablet_replica.cc:333] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:41.917690 29566 raft_consensus.cc:2243] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:41.918159 29566 raft_consensus.cc:2272] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.918884 29566 ts_tablet_manager.cc:1916] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.923852 28383 tablet_service.cc:1511] Processing CreateTablet for tablet a6ba4976bb6e4c2da3a563ccad4eb2b0 (DEFAULT_TABLE table=default.Table_dba55442642c45b1b8dfe959d62699c9 [id=596cac3f4f6540019b75bb586ec7f9ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:41.923852 28644 tablet_service.cc:1511] Processing CreateTablet for tablet a6ba4976bb6e4c2da3a563ccad4eb2b0 (DEFAULT_TABLE table=default.Table_dba55442642c45b1b8dfe959d62699c9 [id=596cac3f4f6540019b75bb586ec7f9ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:41.924226 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6ba4976bb6e4c2da3a563ccad4eb2b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:41.924225 28644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6ba4976bb6e4c2da3a563ccad4eb2b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:41.925029 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 4cb78c3ca3b74b399d2b11320473e463 (DEFAULT_TABLE table=default.Table_6492eae424c54e20b41a06cd5db24684 [id=40a18c6c8416481aa959f8c0c1717db5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:41.925339 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cb78c3ca3b74b399d2b11320473e463. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:41.930816 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 4cb78c3ca3b74b399d2b11320473e463 (DEFAULT_TABLE table=default.Table_6492eae424c54e20b41a06cd5db24684 [id=40a18c6c8416481aa959f8c0c1717db5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:41.931214 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cb78c3ca3b74b399d2b11320473e463. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:41.937862 28797 tablet_bootstrap.cc:492] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:41.938033 28797 ts_tablet_manager.cc:1403] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.191s user 0.007s sys 0.008s
I20260618 13:59:41.938836 28797 raft_consensus.cc:359] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.939008 28797 raft_consensus.cc:385] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.939073 28797 raft_consensus.cc:740] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.939285 28797 consensus_queue.cc:260] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.939802 28797 ts_tablet_manager.cc:1434] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260618 13:59:41.940104 28797 tablet_bootstrap.cc:492] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:41.940122 28799 tablet_bootstrap.cc:654] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.924383 28518 tablet_service.cc:1511] Processing CreateTablet for tablet 4cb78c3ca3b74b399d2b11320473e463 (DEFAULT_TABLE table=default.Table_6492eae424c54e20b41a06cd5db24684 [id=40a18c6c8416481aa959f8c0c1717db5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:41.927635 28528 tablet_service.cc:1511] Processing CreateTablet for tablet a6ba4976bb6e4c2da3a563ccad4eb2b0 (DEFAULT_TABLE table=default.Table_dba55442642c45b1b8dfe959d62699c9 [id=596cac3f4f6540019b75bb586ec7f9ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:41.936873 30551 raft_consensus.cc:493] T 2d1073b726ef4cbeac6c874fc8863e90 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.941581 28797 tablet_bootstrap.cc:654] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.941684 30551 raft_consensus.cc:515] T 2d1073b726ef4cbeac6c874fc8863e90 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.942274 30551 leader_election.cc:290] T 2d1073b726ef4cbeac6c874fc8863e90 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:41.942951 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d1073b726ef4cbeac6c874fc8863e90" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.942943 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d1073b726ef4cbeac6c874fc8863e90" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:41.944478 28483 leader_election.cc:304] T 2d1073b726ef4cbeac6c874fc8863e90 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:41.944859 30551 raft_consensus.cc:2749] T 2d1073b726ef4cbeac6c874fc8863e90 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:41.940698 28798 tablet_bootstrap.cc:492] T 054a7afb3b644aa49df78302cf8f02c9 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:41.946002 28798 ts_tablet_manager.cc:1403] T 054a7afb3b644aa49df78302cf8f02c9 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.155s user 0.001s sys 0.002s
I20260618 13:59:41.946714 28798 raft_consensus.cc:359] T 054a7afb3b644aa49df78302cf8f02c9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.946874 28798 raft_consensus.cc:385] T 054a7afb3b644aa49df78302cf8f02c9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.946962 28798 raft_consensus.cc:740] T 054a7afb3b644aa49df78302cf8f02c9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.947170 28798 consensus_queue.cc:260] T 054a7afb3b644aa49df78302cf8f02c9 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.945308 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6ba4976bb6e4c2da3a563ccad4eb2b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:41.947877 28798 ts_tablet_manager.cc:1434] T 054a7afb3b644aa49df78302cf8f02c9 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:41.948063 28518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cb78c3ca3b74b399d2b11320473e463. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:41.957628 29566 ts_tablet_manager.cc:1929] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:41.957743 29566 log.cc:1199] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/f278d0acbb0c4a0ca61640bc142e6335
I20260618 13:59:41.961879 28797 tablet_bootstrap.cc:492] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:41.962026 28797 ts_tablet_manager.cc:1403] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.004s
I20260618 13:59:41.962728 28797 raft_consensus.cc:359] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.962898 28797 raft_consensus.cc:385] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.962954 28797 raft_consensus.cc:740] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.963183 28797 consensus_queue.cc:260] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.964103 28797 ts_tablet_manager.cc:1434] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:41.964968 28797 tablet_bootstrap.cc:492] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:41.966531 29746 raft_consensus.cc:2243] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:41.966759 29746 raft_consensus.cc:2272] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.966790 28798 tablet_bootstrap.cc:492] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:41.968488 28798 tablet_bootstrap.cc:654] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.970495 28797 tablet_bootstrap.cc:654] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:41.971608 29566 ts_tablet_manager.cc:1950] T f278d0acbb0c4a0ca61640bc142e6335 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:41.972168 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 124) took 9558 ms. Trace:
I20260618 13:59:41.972263 29566 rpcz_store.cc:276] 0618 13:59:32.413816 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.415036 (+ 1220us) service_pool.cc:224] Handling call
0618 13:59:41.917273 (+9502237us) ts_tablet_manager.cc:1151] Deleting tablet f278d0acbb0c4a0ca61640bc142e6335
0618 13:59:41.917289 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:41.922519 (+ 5230us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.957574 (+ 35055us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:41.972148 (+ 14574us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9502023}
I20260618 13:59:41.974237 30623 raft_consensus.cc:493] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:41.974376 30623 raft_consensus.cc:515] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.974913 29566 tablet_replica.cc:333] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:41.975000 30623 leader_election.cc:290] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.975224 29566 raft_consensus.cc:2243] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:41.975497 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f5d4296952e48a48756e6f27140da87" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:41.975570 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f5d4296952e48a48756e6f27140da87" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:41.975742 28682 raft_consensus.cc:2468] T 3f5d4296952e48a48756e6f27140da87 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
W20260618 13:59:41.975989 28346 leader_election.cc:343] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.976142 28346 leader_election.cc:304] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:41.976518 30623 raft_consensus.cc:2804] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:41.976579 30623 raft_consensus.cc:493] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:41.976637 30623 raft_consensus.cc:3060] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.975560 29566 raft_consensus.cc:2272] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:41.977677 29566 ts_tablet_manager.cc:1916] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:41.979183 30623 raft_consensus.cc:515] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.979660 30623 leader_election.cc:290] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:41.980528 28798 tablet_bootstrap.cc:492] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:41.980573 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f5d4296952e48a48756e6f27140da87" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:41.980649 28798 ts_tablet_manager.cc:1403] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.000s
I20260618 13:59:41.980703 28682 raft_consensus.cc:3060] T 3f5d4296952e48a48756e6f27140da87 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:41.980679 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f5d4296952e48a48756e6f27140da87" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:41.981181 28346 leader_election.cc:343] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:41.981321 28798 raft_consensus.cc:359] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.981504 28798 raft_consensus.cc:385] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:41.981572 28798 raft_consensus.cc:740] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:41.981793 28798 consensus_queue.cc:260] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.982404 28798 ts_tablet_manager.cc:1434] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:41.982714 28798 tablet_bootstrap.cc:492] T d2d54a901e9f4d5c89700979ce0618bf P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:41.983565 28682 raft_consensus.cc:2468] T 3f5d4296952e48a48756e6f27140da87 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:41.984606 28346 leader_election.cc:304] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:41.985558 30623 raft_consensus.cc:2804] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:41.985675 30623 raft_consensus.cc:697] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:41.985900 30623 consensus_queue.cc:237] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:41.986559 29746 ts_tablet_manager.cc:1916] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.004688 28798 tablet_bootstrap.cc:654] T d2d54a901e9f4d5c89700979ce0618bf P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.006292 30262 raft_consensus.cc:493] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.006520 30262 raft_consensus.cc:515] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.007092 30262 leader_election.cc:290] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:42.007975 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f2d02ec004b9298b94307c7a38a97" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:42.009122 28346 leader_election.cc:343] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.007735 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f2d02ec004b9298b94307c7a38a97" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:42.014011 28346 leader_election.cc:343] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.014140 28346 leader_election.cc:304] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:42.015247 30262 raft_consensus.cc:2749] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:42.019270 28799 tablet_bootstrap.cc:492] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:42.019815 28799 ts_tablet_manager.cc:1403] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.201s user 0.001s sys 0.002s
I20260618 13:59:42.021060 28799 raft_consensus.cc:359] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.021410 28799 raft_consensus.cc:385] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.021484 28799 raft_consensus.cc:740] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.021733 28799 consensus_queue.cc:260] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.022248 28799 ts_tablet_manager.cc:1434] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:42.032697 29746 ts_tablet_manager.cc:1929] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:42.032860 29746 log.cc:1199] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/0463fa2e30554abcbfc8f37b702e81d4
I20260618 13:59:42.033661 28798 tablet_bootstrap.cc:492] T d2d54a901e9f4d5c89700979ce0618bf P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:42.033861 28798 ts_tablet_manager.cc:1403] T d2d54a901e9f4d5c89700979ce0618bf P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.051s user 0.000s sys 0.003s
I20260618 13:59:42.034611 28798 raft_consensus.cc:359] T d2d54a901e9f4d5c89700979ce0618bf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.034853 28798 raft_consensus.cc:385] T d2d54a901e9f4d5c89700979ce0618bf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.034945 28798 raft_consensus.cc:740] T d2d54a901e9f4d5c89700979ce0618bf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.035190 28798 consensus_queue.cc:260] T d2d54a901e9f4d5c89700979ce0618bf P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.037837 28798 ts_tablet_manager.cc:1434] T d2d54a901e9f4d5c89700979ce0618bf P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260618 13:59:42.046495 30551 raft_consensus.cc:493] T d2d54a901e9f4d5c89700979ce0618bf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.057722 30551 raft_consensus.cc:515] T d2d54a901e9f4d5c89700979ce0618bf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.058287 30551 leader_election.cc:290] T d2d54a901e9f4d5c89700979ce0618bf P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.058945 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2d54a901e9f4d5c89700979ce0618bf" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:42.059422 28799 tablet_bootstrap.cc:492] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:42.062153 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2d54a901e9f4d5c89700979ce0618bf" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:42.063020 28484 leader_election.cc:304] T d2d54a901e9f4d5c89700979ce0618bf P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:42.063863 30551 raft_consensus.cc:2749] T d2d54a901e9f4d5c89700979ce0618bf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:42.065865 28797 tablet_bootstrap.cc:492] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:42.066011 28797 ts_tablet_manager.cc:1403] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.101s user 0.001s sys 0.002s
I20260618 13:59:42.066695 28797 raft_consensus.cc:359] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.066859 28797 raft_consensus.cc:385] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.066919 28797 raft_consensus.cc:740] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.067173 28797 consensus_queue.cc:260] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.056224 29746 ts_tablet_manager.cc:1950] T 0463fa2e30554abcbfc8f37b702e81d4 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:42.071102 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 151) took 7071 ms. Trace:
I20260618 13:59:42.071185 29746 rpcz_store.cc:276] 0618 13:59:34.999483 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.999967 (+ 484us) service_pool.cc:224] Handling call
0618 13:59:41.907592 (+6907625us) ts_tablet_manager.cc:1151] Deleting tablet 0463fa2e30554abcbfc8f37b702e81d4
0618 13:59:41.907606 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.011787 (+104181us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.032632 (+ 20845us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.071086 (+ 38454us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6850985}
I20260618 13:59:42.071632 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 0463fa2e30554abcbfc8f37b702e81d4 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]) successfully deleted
I20260618 13:59:42.078877 29746 tablet_replica.cc:333] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:42.078997 28797 ts_tablet_manager.cc:1434] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260618 13:59:42.079231 29746 raft_consensus.cc:2243] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:42.079725 30572 raft_consensus.cc:493] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.079841 30572 raft_consensus.cc:515] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:42.080331 29746 raft_consensus.cc:2272] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.080461 30572 leader_election.cc:290] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.080919 29746 ts_tablet_manager.cc:1916] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:42.082221 28617 leader_election.cc:343] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 962bf304a285412fb2a529db5b994e9e
I20260618 13:59:42.082381 28797 tablet_bootstrap.cc:492] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:42.082547 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "962bf304a285412fb2a529db5b994e9e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:42.083009 28618 leader_election.cc:343] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.083104 28618 leader_election.cc:304] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:42.083510 30577 raft_consensus.cc:2749] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:42.086975 29746 ts_tablet_manager.cc:1929] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:42.087106 29746 log.cc:1199] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/125bb6885ab348d885d5a3b4bdd45bc7
I20260618 13:59:42.087466 29746 ts_tablet_manager.cc:1950] T 125bb6885ab348d885d5a3b4bdd45bc7 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:42.088524 28797 tablet_bootstrap.cc:654] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.057617 28798 tablet_bootstrap.cc:492] T c38a2156a55e4e5eb91d101e9a61eecd P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:42.090078 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 146) took 7091 ms. Trace:
I20260618 13:59:42.090164 29746 rpcz_store.cc:276] 0618 13:59:34.998941 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.999865 (+ 924us) service_pool.cc:224] Handling call
0618 13:59:42.078822 (+7078957us) ts_tablet_manager.cc:1151] Deleting tablet 125bb6885ab348d885d5a3b4bdd45bc7
0618 13:59:42.078833 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.083929 (+ 5096us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.086930 (+ 3001us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.090062 (+ 3132us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7022174}
I20260618 13:59:42.090677 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 125bb6885ab348d885d5a3b4bdd45bc7 (table default.Table_af38438888374cf2b0b8f3db85810efc [id=84cf9477c8584508b8e1c847cdf58b28]) successfully deleted
I20260618 13:59:42.091569 28546 raft_consensus.cc:3060] T 80e99a85e83c46afa367e59e5ddb7dc7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.091653 28798 tablet_bootstrap.cc:654] T c38a2156a55e4e5eb91d101e9a61eecd P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.092619 29746 tablet_replica.cc:333] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:42.092794 29746 raft_consensus.cc:2243] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:42.092941 29746 raft_consensus.cc:2272] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.093425 29746 ts_tablet_manager.cc:1916] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.097025 28730 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
W20260618 13:59:42.098997 28617 consensus_peers.cc:597] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b0d3d363766546c4a97b0763bad11f71. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:42.099049 29746 ts_tablet_manager.cc:1929] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:42.099156 29746 log.cc:1199] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/be59f988f0e34b9480f859cf88b9c223
I20260618 13:59:42.099545 29746 ts_tablet_manager.cc:1950] T be59f988f0e34b9480f859cf88b9c223 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:42.101318 29566 ts_tablet_manager.cc:1929] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:42.101524 29566 log.cc:1199] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/d3dbf13544ce408ab97834aec5862390
I20260618 13:59:42.103246 29566 ts_tablet_manager.cc:1950] T d3dbf13544ce408ab97834aec5862390 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:42.106775 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 128) took 9547 ms. Trace:
I20260618 13:59:42.107117 28799 tablet_bootstrap.cc:654] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.110459 28798 tablet_bootstrap.cc:492] T c38a2156a55e4e5eb91d101e9a61eecd P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:42.110690 28798 ts_tablet_manager.cc:1403] T c38a2156a55e4e5eb91d101e9a61eecd P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.053s user 0.003s sys 0.000s
I20260618 13:59:42.112393 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 152) took 7112 ms. Trace:
I20260618 13:59:42.112484 29746 rpcz_store.cc:276] 0618 13:59:34.999593 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.002175 (+ 2582us) service_pool.cc:224] Handling call
0618 13:59:42.092575 (+7090400us) ts_tablet_manager.cc:1151] Deleting tablet be59f988f0e34b9480f859cf88b9c223
0618 13:59:42.092585 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.096390 (+ 3805us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.098995 (+ 2605us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.112372 (+ 13377us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11891,"tablet-delete.queue_time_us":7035886}
I20260618 13:59:42.111491 28798 raft_consensus.cc:359] T c38a2156a55e4e5eb91d101e9a61eecd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:42.112941 28798 raft_consensus.cc:385] T c38a2156a55e4e5eb91d101e9a61eecd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.113054 28798 raft_consensus.cc:740] T c38a2156a55e4e5eb91d101e9a61eecd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.113198 29746 tablet_replica.cc:333] T 6eea4d59935d40aa97dc10ec7469aa56 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:42.113418 29746 raft_consensus.cc:2243] T 6eea4d59935d40aa97dc10ec7469aa56 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:42.113380 28798 consensus_queue.cc:260] T c38a2156a55e4e5eb91d101e9a61eecd P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:42.115128 28537 raft_consensus.cc:3060] T c38a2156a55e4e5eb91d101e9a61eecd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.113581 29746 raft_consensus.cc:2272] T 6eea4d59935d40aa97dc10ec7469aa56 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.117626 29746 ts_tablet_manager.cc:1916] T 6eea4d59935d40aa97dc10ec7469aa56 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.121234 29566 rpcz_store.cc:276] 0618 13:59:32.558921 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.558970 (+ 49us) service_pool.cc:224] Handling call
0618 13:59:41.974843 (+9415873us) ts_tablet_manager.cc:1151] Deleting tablet d3dbf13544ce408ab97834aec5862390
0618 13:59:41.974856 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.058361 (+ 83505us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.101262 (+ 42901us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.106758 (+ 5496us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9415654}
W20260618 13:59:42.118950 28617 consensus_peers.cc:597] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f2b21dcf74fa4264947ca43cc0026e76. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:42.118518 28797 tablet_bootstrap.cc:492] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:42.123569 28797 ts_tablet_manager.cc:1403] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.044s user 0.001s sys 0.003s
I20260618 13:59:42.123910 29746 ts_tablet_manager.cc:1929] T 6eea4d59935d40aa97dc10ec7469aa56 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:42.124029 29746 log.cc:1199] T 6eea4d59935d40aa97dc10ec7469aa56 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/6eea4d59935d40aa97dc10ec7469aa56
I20260618 13:59:42.125298 28797 raft_consensus.cc:359] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.125548 28797 raft_consensus.cc:385] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.125648 28797 raft_consensus.cc:740] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.125936 28797 consensus_queue.cc:260] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.128981 30622 raft_consensus.cc:493] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.129125 30622 raft_consensus.cc:515] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.129778 30622 leader_election.cc:290] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:42.130733 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e438b3733c374e1bb8aae4ed9f16b4a4" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:42.131592 28346 leader_election.cc:343] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.124403 29746 ts_tablet_manager.cc:1950] T 6eea4d59935d40aa97dc10ec7469aa56 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:42.135066 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 150) took 7135 ms. Trace:
I20260618 13:59:42.135164 29746 rpcz_store.cc:276] 0618 13:59:34.999381 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.999945 (+ 564us) service_pool.cc:224] Handling call
0618 13:59:42.113148 (+7113203us) ts_tablet_manager.cc:1151] Deleting tablet 6eea4d59935d40aa97dc10ec7469aa56
0618 13:59:42.113159 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.121073 (+ 7914us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.123858 (+ 2785us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.135044 (+ 11186us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7056413}
I20260618 13:59:42.135646 29746 tablet_replica.cc:333] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:42.137457 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e438b3733c374e1bb8aae4ed9f16b4a4" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:42.137982 28346 leader_election.cc:343] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.138092 28346 leader_election.cc:304] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:42.138577 30622 raft_consensus.cc:2749] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:42.140020 28346 consensus_peers.cc:597] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:42.140340 28797 ts_tablet_manager.cc:1434] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.016s user 0.002s sys 0.000s
I20260618 13:59:42.144902 29566 tablet_replica.cc:333] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:42.145198 29566 raft_consensus.cc:2243] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:42.149731 29746 raft_consensus.cc:2243] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:42.149914 28798 ts_tablet_manager.cc:1434] T c38a2156a55e4e5eb91d101e9a61eecd P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.039s user 0.002s sys 0.000s
I20260618 13:59:42.149940 29746 raft_consensus.cc:2272] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.150385 28798 tablet_bootstrap.cc:492] T fc1968e5ce2e42a5b836c90b0904f158 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
W20260618 13:59:42.153436 28346 consensus_peers.cc:597] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:42.154321 29566 raft_consensus.cc:2272] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.155205 29566 ts_tablet_manager.cc:1916] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.158147 28799 tablet_bootstrap.cc:492] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:42.158337 28799 ts_tablet_manager.cc:1403] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.099s user 0.003s sys 0.000s
I20260618 13:59:42.159210 28799 raft_consensus.cc:359] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.159480 28799 raft_consensus.cc:385] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.159596 28799 raft_consensus.cc:740] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.160787 28799 consensus_queue.cc:260] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.166600 28799 ts_tablet_manager.cc:1434] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260618 13:59:42.166976 28799 tablet_bootstrap.cc:492] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:42.150650 29746 ts_tablet_manager.cc:1916] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.170291 28798 tablet_bootstrap.cc:654] T fc1968e5ce2e42a5b836c90b0904f158 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.171949 28799 tablet_bootstrap.cc:654] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.172976 28546 raft_consensus.cc:3060] T 340f6bbf18eb412ca3afec237ff7752d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.174501 28797 tablet_bootstrap.cc:492] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:42.174826 29746 ts_tablet_manager.cc:1929] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:42.174958 29746 log.cc:1199] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/02b5cc08ca0c4e719e0414b345211ba5
I20260618 13:59:42.175330 29746 ts_tablet_manager.cc:1950] T 02b5cc08ca0c4e719e0414b345211ba5 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:42.178905 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 149) took 7179 ms. Trace:
I20260618 13:59:42.179035 29746 rpcz_store.cc:276] 0618 13:59:34.999264 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.999922 (+ 658us) service_pool.cc:224] Handling call
0618 13:59:42.135596 (+7135674us) ts_tablet_manager.cc:1151] Deleting tablet 02b5cc08ca0c4e719e0414b345211ba5
0618 13:59:42.135607 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.171981 (+ 36374us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.174775 (+ 2794us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.178880 (+ 4105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15860,"tablet-delete.queue_time_us":7078818}
I20260618 13:59:42.179549 29746 tablet_replica.cc:333] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:42.179751 29746 raft_consensus.cc:2243] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:42.179903 29746 raft_consensus.cc:2272] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.180604 29746 ts_tablet_manager.cc:1916] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.189160 30572 consensus_queue.cc:1048] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:42.193703 29746 ts_tablet_manager.cc:1929] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:42.193869 29746 log.cc:1199] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/c7f42997416841a7b2c1ad347d2e9282
I20260618 13:59:42.196830 28799 tablet_bootstrap.cc:492] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:42.197000 28799 ts_tablet_manager.cc:1403] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.009s
I20260618 13:59:42.197826 28799 raft_consensus.cc:359] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.198035 28799 raft_consensus.cc:385] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.198096 28799 raft_consensus.cc:740] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.113051 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet be59f988f0e34b9480f859cf88b9c223 (table default.Table_b8741a7fd29f4e78a948b1c419595a85 [id=6f4d908593ae40f09e2c94cef218e1fd]) successfully deleted
W20260618 13:59:42.198616 28989 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 85597.1us
I20260618 13:59:42.198987 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet d3dbf13544ce408ab97834aec5862390 (table default.Table_bf5ce6c6e0b64002994fb45e76ab30eb [id=a0e6904d88704db39c194c34ac9a7faf]) successfully deleted
I20260618 13:59:42.194380 29746 ts_tablet_manager.cc:1950] T c7f42997416841a7b2c1ad347d2e9282 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:42.199503 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 6eea4d59935d40aa97dc10ec7469aa56 (table default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3]) successfully deleted
I20260618 13:59:42.199424 28799 consensus_queue.cc:260] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.199728 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 02b5cc08ca0c4e719e0414b345211ba5 (table default.Table_6f3c7c1d308e4901887fc9b602b38e4f [id=7aa3803c01134af6b073952eb9843282]) successfully deleted
I20260618 13:59:42.200105 28799 ts_tablet_manager.cc:1434] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:42.200410 28799 tablet_bootstrap.cc:492] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:42.200405 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 144) took 7201 ms. Trace:
I20260618 13:59:42.200538 30622 consensus_queue.cc:1048] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:42.200491 29746 rpcz_store.cc:276] 0618 13:59:34.998688 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.999825 (+ 1137us) service_pool.cc:224] Handling call
0618 13:59:42.179501 (+7179676us) ts_tablet_manager.cc:1151] Deleting tablet c7f42997416841a7b2c1ad347d2e9282
0618 13:59:42.179512 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.183688 (+ 4176us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.193620 (+ 9932us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.200380 (+ 6760us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7122683}
I20260618 13:59:42.200999 29746 tablet_replica.cc:333] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:42.200973 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet c7f42997416841a7b2c1ad347d2e9282 (table default.Table_0a13b90182654346acbbc8f47a56436d [id=c31d65adb55f458c9ced14d598464f89]) successfully deleted
I20260618 13:59:42.201203 29746 raft_consensus.cc:2243] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:42.201576 29746 raft_consensus.cc:2272] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.202181 29746 ts_tablet_manager.cc:1916] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.203147 28799 tablet_bootstrap.cc:654] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.098369 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet ba53ad1988064049a61fcca3a85f8779 (table default.Table_b7f097c1abe1410a800ce5aa25771740 [id=6c30b030a60b47e2be48dfe6e9cfdfdb]). Expected version 1 got 0
I20260618 13:59:42.205732 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:42.213980 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:42.215171 28797 tablet_bootstrap.cc:654] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.215553 29746 ts_tablet_manager.cc:1929] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:42.215677 29746 log.cc:1199] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/c552d1316e3340f4bcfa39618ae309b8
I20260618 13:59:42.216063 29746 ts_tablet_manager.cc:1950] T c552d1316e3340f4bcfa39618ae309b8 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:42.216866 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 147) took 7217 ms. Trace:
I20260618 13:59:42.216969 29746 rpcz_store.cc:276] 0618 13:59:34.999051 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.999893 (+ 842us) service_pool.cc:224] Handling call
0618 13:59:42.200942 (+7201049us) ts_tablet_manager.cc:1151] Deleting tablet c552d1316e3340f4bcfa39618ae309b8
0618 13:59:42.200954 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.211540 (+ 10586us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.215492 (+ 3952us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.216843 (+ 1351us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7144078}
I20260618 13:59:42.217411 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 168) took 7158 ms. Trace:
I20260618 13:59:42.217515 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet c552d1316e3340f4bcfa39618ae309b8 (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]) successfully deleted
I20260618 13:59:42.217507 29746 rpcz_store.cc:276] 0618 13:59:35.058584 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.058635 (+ 51us) service_pool.cc:224] Handling call
0618 13:59:42.217259 (+7158624us) ts_tablet_manager.cc:1151] Deleting tablet e3d8616846c649b6a969b971db0d03db
0618 13:59:42.217274 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.217392 (+ 118us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7158479}
I20260618 13:59:42.217818 29746 tablet_replica.cc:333] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:42.218022 29746 raft_consensus.cc:2243] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260618 13:59:42.218142 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet e3d8616846c649b6a969b971db0d03db because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e3d8616846c649b6a969b971db0d03db already in progress: creating tablet
I20260618 13:59:42.218199 29746 raft_consensus.cc:2272] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.218088 29023 catalog_manager.cc:5696] T fc1968e5ce2e42a5b836c90b0904f158 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:42.218771 29746 ts_tablet_manager.cc:1916] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.218714 29023 catalog_manager.cc:5696] T 85b2f34bc17b4a27a91d6cafa9a96c48 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:42.231519 28798 tablet_bootstrap.cc:492] T fc1968e5ce2e42a5b836c90b0904f158 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
W20260618 13:59:42.231693 28618 consensus_peers.cc:597] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:42.231756 28798 ts_tablet_manager.cc:1403] T fc1968e5ce2e42a5b836c90b0904f158 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.081s user 0.003s sys 0.000s
I20260618 13:59:42.232582 28798 raft_consensus.cc:359] T fc1968e5ce2e42a5b836c90b0904f158 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.232852 28798 raft_consensus.cc:385] T fc1968e5ce2e42a5b836c90b0904f158 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.232957 28798 raft_consensus.cc:740] T fc1968e5ce2e42a5b836c90b0904f158 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.233217 28798 consensus_queue.cc:260] T fc1968e5ce2e42a5b836c90b0904f158 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.235548 28798 ts_tablet_manager.cc:1434] T fc1968e5ce2e42a5b836c90b0904f158 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260618 13:59:42.236968 28798 tablet_bootstrap.cc:492] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:42.237502 29746 ts_tablet_manager.cc:1929] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:42.237627 29746 log.cc:1199] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/cd06bb7cffc34412b420d70435fce373
I20260618 13:59:42.238006 29746 ts_tablet_manager.cc:1950] T cd06bb7cffc34412b420d70435fce373 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
W20260618 13:59:42.241485 28346 consensus_peers.cc:597] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:42.242066 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 169) took 7181 ms. Trace:
I20260618 13:59:42.242170 29746 rpcz_store.cc:276] 0618 13:59:35.060655 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.060700 (+ 45us) service_pool.cc:224] Handling call
0618 13:59:42.217758 (+7157058us) ts_tablet_manager.cc:1151] Deleting tablet cd06bb7cffc34412b420d70435fce373
0618 13:59:42.217770 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.221695 (+ 3925us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.237441 (+ 15746us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.242044 (+ 4603us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7156890}
I20260618 13:59:42.243407 29746 tablet_replica.cc:333] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:42.244292 28797 tablet_bootstrap.cc:492] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:42.244513 28797 ts_tablet_manager.cc:1403] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.094s user 0.002s sys 0.002s
I20260618 13:59:42.245369 28797 raft_consensus.cc:359] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.245654 29746 raft_consensus.cc:2243] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:42.245630 28797 raft_consensus.cc:385] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.245848 28797 raft_consensus.cc:740] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.245869 29746 raft_consensus.cc:2272] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.246469 29746 ts_tablet_manager.cc:1916] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:42.246522 28618 consensus_peers.cc:597] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:42.246470 28797 consensus_queue.cc:260] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.247260 28797 ts_tablet_manager.cc:1434] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260618 13:59:42.247622 28797 tablet_bootstrap.cc:492] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:42.249496 28797 tablet_bootstrap.cc:654] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.242683 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet cd06bb7cffc34412b420d70435fce373 (table default.Table_74fc7c21dc664ff8bfe0d25aee399bdb [id=d224150fd3a340fc96e75c0b96146b11]) successfully deleted
I20260618 13:59:42.250133 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:42.252874 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:42.253191 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:42.253523 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_01e1e5741bb54f78996857f6b79fc5d7 [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 2 got 0
I20260618 13:59:42.255621 28537 raft_consensus.cc:3060] T 2d1073b726ef4cbeac6c874fc8863e90 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.257496 28799 tablet_bootstrap.cc:492] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:42.257643 28799 ts_tablet_manager.cc:1403] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.057s user 0.008s sys 0.009s
I20260618 13:59:42.258524 28799 raft_consensus.cc:359] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.258738 28799 raft_consensus.cc:385] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.258808 28799 raft_consensus.cc:740] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.259043 28799 consensus_queue.cc:260] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.259615 28799 ts_tablet_manager.cc:1434] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:42.259933 28799 tablet_bootstrap.cc:492] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:42.262823 30622 consensus_queue.cc:1048] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:42.263072 28799 tablet_bootstrap.cc:654] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.263296 29746 ts_tablet_manager.cc:1929] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:42.263414 29746 log.cc:1199] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/f278d0acbb0c4a0ca61640bc142e6335
I20260618 13:59:42.263772 29746 ts_tablet_manager.cc:1950] T f278d0acbb0c4a0ca61640bc142e6335 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:42.264596 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 170) took 7201 ms. Trace:
I20260618 13:59:42.264693 29746 rpcz_store.cc:276] 0618 13:59:35.062838 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.062882 (+ 44us) service_pool.cc:224] Handling call
0618 13:59:42.243354 (+7180472us) ts_tablet_manager.cc:1151] Deleting tablet f278d0acbb0c4a0ca61640bc142e6335
0618 13:59:42.243365 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.249567 (+ 6202us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.263231 (+ 13664us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.264573 (+ 1342us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7180325}
I20260618 13:59:42.267107 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 174) took 6825 ms. Trace:
I20260618 13:59:42.267206 29746 rpcz_store.cc:276] 0618 13:59:35.441776 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.441831 (+ 55us) service_pool.cc:224] Handling call
0618 13:59:42.266952 (+6825121us) ts_tablet_manager.cc:1151] Deleting tablet f45556a59c424ca2bc30956566374c89
0618 13:59:42.266963 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.267088 (+ 125us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6824852}
I20260618 13:59:42.270680 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 176) took 6576 ms. Trace:
I20260618 13:59:42.270771 29746 rpcz_store.cc:276] 0618 13:59:35.694166 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.696463 (+ 2297us) service_pool.cc:224] Handling call
0618 13:59:42.270550 (+6574087us) ts_tablet_manager.cc:1151] Deleting tablet 98446f40214c4ac984fcfe98310005c5
0618 13:59:42.270563 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.270664 (+ 101us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6570531}
I20260618 13:59:42.271600 29746 tablet_replica.cc:333] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:42.271833 29746 raft_consensus.cc:2243] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:42.272003 29746 raft_consensus.cc:2272] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.272521 29746 ts_tablet_manager.cc:1916] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.278352 29746 ts_tablet_manager.cc:1929] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:42.281555 28799 tablet_bootstrap.cc:492] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:42.281700 28799 ts_tablet_manager.cc:1403] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20260618 13:59:42.282549 28799 raft_consensus.cc:359] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.282744 28799 raft_consensus.cc:385] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.282809 28799 raft_consensus.cc:740] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.283008 28799 consensus_queue.cc:260] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.291626 28799 ts_tablet_manager.cc:1434] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260618 13:59:42.291954 28799 tablet_bootstrap.cc:492] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:42.254308 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:42.294515 28799 tablet_bootstrap.cc:654] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:42.267692 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet f45556a59c424ca2bc30956566374c89 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f45556a59c424ca2bc30956566374c89 already in progress: creating tablet
W20260618 13:59:42.295297 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 98446f40214c4ac984fcfe98310005c5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 98446f40214c4ac984fcfe98310005c5 already in progress: creating tablet
I20260618 13:59:42.295400 29746 log.cc:1199] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/cea4bc13eabc4dd98b4f96723498014c
I20260618 13:59:42.295809 29746 ts_tablet_manager.cc:1950] T cea4bc13eabc4dd98b4f96723498014c P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:42.296586 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 188) took 4885 ms. Trace:
I20260618 13:59:42.296674 29746 rpcz_store.cc:276] 0618 13:59:37.410968 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:37.411430 (+ 462us) service_pool.cc:224] Handling call
0618 13:59:42.271553 (+4860123us) ts_tablet_manager.cc:1151] Deleting tablet cea4bc13eabc4dd98b4f96723498014c
0618 13:59:42.271562 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.275595 (+ 4033us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.278303 (+ 2708us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.296565 (+ 18262us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4859929}
I20260618 13:59:42.297106 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 189) took 4798 ms. Trace:
I20260618 13:59:42.297184 29746 rpcz_store.cc:276] 0618 13:59:37.498866 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:37.498917 (+ 51us) service_pool.cc:224] Handling call
0618 13:59:42.296978 (+4798061us) ts_tablet_manager.cc:1151] Deleting tablet c01c8463055e4526b3c75190066f30b6
0618 13:59:42.296989 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.297089 (+ 100us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4797886}
I20260618 13:59:42.297502 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 190) took 4776 ms. Trace:
I20260618 13:59:42.297570 29746 rpcz_store.cc:276] 0618 13:59:37.521107 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:37.521157 (+ 50us) service_pool.cc:224] Handling call
0618 13:59:42.297409 (+4776252us) ts_tablet_manager.cc:1151] Deleting tablet 6c4a31a4f202473d97b17b5b5dfdb615
0618 13:59:42.297418 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.297488 (+ 70us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4776086}
I20260618 13:59:42.297817 29746 tablet_replica.cc:333] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:42.298022 29746 raft_consensus.cc:2243] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:42.298164 29746 raft_consensus.cc:2272] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.298164 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet cea4bc13eabc4dd98b4f96723498014c (table default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c]) successfully deleted
W20260618 13:59:42.298441 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet c01c8463055e4526b3c75190066f30b6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c01c8463055e4526b3c75190066f30b6 already in progress: creating tablet
W20260618 13:59:42.298699 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 6c4a31a4f202473d97b17b5b5dfdb615 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6c4a31a4f202473d97b17b5b5dfdb615 already in progress: creating tablet
I20260618 13:59:42.298722 29746 ts_tablet_manager.cc:1916] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.299610 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:42.300235 28798 tablet_bootstrap.cc:654] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.304410 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:42.312318 29746 ts_tablet_manager.cc:1929] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:42.312461 29746 log.cc:1199] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/18d8b609b96a4e81b57f69bff522eeea
I20260618 13:59:42.312817 29746 ts_tablet_manager.cc:1950] T 18d8b609b96a4e81b57f69bff522eeea P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:42.313657 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 192) took 2386 ms. Trace:
I20260618 13:59:42.313755 29746 rpcz_store.cc:276] 0618 13:59:39.927403 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.927664 (+ 261us) service_pool.cc:224] Handling call
0618 13:59:42.297767 (+2370103us) ts_tablet_manager.cc:1151] Deleting tablet 18d8b609b96a4e81b57f69bff522eeea
0618 13:59:42.297775 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.301756 (+ 3981us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.312248 (+ 10492us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.313635 (+ 1387us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2369901}
I20260618 13:59:42.314139 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 194) took 2379 ms. Trace:
I20260618 13:59:42.314229 29746 rpcz_store.cc:276] 0618 13:59:39.934173 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.934480 (+ 307us) service_pool.cc:224] Handling call
0618 13:59:42.314052 (+2379572us) ts_tablet_manager.cc:1151] Deleting tablet 3a0039a463ec4a9c82a21a3a69b565fc
0618 13:59:42.314064 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.314125 (+ 61us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2379318}
I20260618 13:59:42.302645 29566 ts_tablet_manager.cc:1929] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:42.324816 29566 log.cc:1199] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/55ae29c3b7ed44cfa659130745037aa6
I20260618 13:59:42.324885 28799 tablet_bootstrap.cc:492] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:42.325034 28799 ts_tablet_manager.cc:1403] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.033s user 0.006s sys 0.002s
I20260618 13:59:42.325352 29566 ts_tablet_manager.cc:1950] T 55ae29c3b7ed44cfa659130745037aa6 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:42.326175 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 145) took 9373 ms. Trace:
I20260618 13:59:42.326249 28799 raft_consensus.cc:359] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.326433 28799 raft_consensus.cc:385] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.326483 28799 raft_consensus.cc:740] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.326328 29566 rpcz_store.cc:276] 0618 13:59:32.952629 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.966156 (+ 13527us) service_pool.cc:224] Handling call
0618 13:59:42.144844 (+9178688us) ts_tablet_manager.cc:1151] Deleting tablet 55ae29c3b7ed44cfa659130745037aa6
0618 13:59:42.144856 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.279701 (+134845us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.302583 (+ 22882us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.326154 (+ 23571us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9178475}
I20260618 13:59:42.326737 28799 consensus_queue.cc:260] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.327221 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 138) took 9429 ms. Trace:
I20260618 13:59:42.327337 29566 rpcz_store.cc:276] 0618 13:59:32.897828 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:32.969735 (+ 71907us) service_pool.cc:224] Handling call
0618 13:59:42.327094 (+9357359us) ts_tablet_manager.cc:1151] Deleting tablet e3d8616846c649b6a969b971db0d03db
0618 13:59:42.327105 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.327204 (+ 99us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9357158}
I20260618 13:59:42.327777 29566 tablet_replica.cc:333] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:42.328027 29566 raft_consensus.cc:2243] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:42.328379 29566 raft_consensus.cc:2272] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.329309 28799 ts_tablet_manager.cc:1434] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 13:59:42.329586 28799 tablet_bootstrap.cc:492] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:42.331027 29566 ts_tablet_manager.cc:1916] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.338349 28799 tablet_bootstrap.cc:654] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.340670 28798 tablet_bootstrap.cc:492] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:42.340894 28798 ts_tablet_manager.cc:1403] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.104s user 0.003s sys 0.000s
I20260618 13:59:42.341616 28798 raft_consensus.cc:359] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:42.341866 28798 raft_consensus.cc:385] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.341958 28798 raft_consensus.cc:740] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.342211 28798 consensus_queue.cc:260] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:42.316676 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 18d8b609b96a4e81b57f69bff522eeea (table default.Table_1b32156053d845a5b192d2ccf583cac6 [id=60305da883dd4da2ab74ac31d5462bf0]) successfully deleted
I20260618 13:59:42.342677 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 55ae29c3b7ed44cfa659130745037aa6 (table default.Table_5d485acde41448d984aa79fa58b1866b [id=726324d365b94ff6bac24030222ca34f]) successfully deleted
W20260618 13:59:42.342948 28989 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet e3d8616846c649b6a969b971db0d03db because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e3d8616846c649b6a969b971db0d03db already in progress: creating tablet
W20260618 13:59:42.343155 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 3a0039a463ec4a9c82a21a3a69b565fc because the tablet was not found. No further retry: Not found: Tablet not found: 3a0039a463ec4a9c82a21a3a69b565fc
I20260618 13:59:42.343719 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:42.344434 29023 catalog_manager.cc:5696] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:42.344945 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:42.345744 29023 catalog_manager.cc:5696] T fe1139c409e84070a2f496a85545e8d5 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:42.346248 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:42.346550 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c9ceb3af08fa4618b81cf6f89d7bd0df (table default.Table_21242eba3cb64c7c91082e296439cfe2 [id=10a4f5ef579746b68d8dd37d83f14c3e]). Expected version 1 got 0
I20260618 13:59:42.347282 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:42.348480 29303 raft_consensus.cc:493] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.348649 29303 raft_consensus.cc:515] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:42.349184 30623 raft_consensus.cc:493] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.349325 30623 raft_consensus.cc:515] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.349922 30623 leader_election.cc:290] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:42.350526 29303 leader_election.cc:290] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.350876 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dffeeb7313714c3f9eccc228c98ae7a3" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:42.342446 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 195) took 2408 ms. Trace:
I20260618 13:59:42.350672 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dffeeb7313714c3f9eccc228c98ae7a3" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:42.351387 28346 leader_election.cc:343] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.351788 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98446f40214c4ac984fcfe98310005c5" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:42.351864 28617 leader_election.cc:343] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 98446f40214c4ac984fcfe98310005c5
W20260618 13:59:42.352146 28346 leader_election.cc:343] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.352250 28346 leader_election.cc:304] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:42.352281 28618 leader_election.cc:343] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.352392 28618 leader_election.cc:304] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:42.352650 29746 rpcz_store.cc:276] 0618 13:59:39.934278 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.934783 (+ 505us) service_pool.cc:224] Handling call
0618 13:59:42.342323 (+2407540us) ts_tablet_manager.cc:1151] Deleting tablet 8b88ed7cf79c4929a8704f66691cbed4
0618 13:59:42.342332 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.342408 (+ 76us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2407375}
I20260618 13:59:42.352985 30584 raft_consensus.cc:2749] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:42.353577 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 193) took 2419 ms. Trace:
I20260618 13:59:42.353657 29746 rpcz_store.cc:276] 0618 13:59:39.934045 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.934978 (+ 933us) service_pool.cc:224] Handling call
0618 13:59:42.353409 (+2418431us) ts_tablet_manager.cc:1151] Deleting tablet 233040a16b0d4b61b977ca57c1afec2d
0618 13:59:42.353419 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.353559 (+ 140us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2418276}
I20260618 13:59:42.353976 30623 raft_consensus.cc:2749] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:42.354351 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 197) took 2415 ms. Trace:
W20260618 13:59:42.354471 28618 consensus_peers.cc:597] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:42.354470 29746 rpcz_store.cc:276] 0618 13:59:39.938861 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.938988 (+ 127us) service_pool.cc:224] Handling call
0618 13:59:42.354263 (+2415275us) ts_tablet_manager.cc:1151] Deleting tablet d7a97176f72349cda221baaea61d18ad
0618 13:59:42.354274 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.354334 (+ 60us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2415075}
I20260618 13:59:42.355000 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 198) took 2416 ms. Trace:
I20260618 13:59:42.355098 29746 rpcz_store.cc:276] 0618 13:59:39.938939 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.939230 (+ 291us) service_pool.cc:224] Handling call
0618 13:59:42.354912 (+2415682us) ts_tablet_manager.cc:1151] Deleting tablet e025b64380314546af8a1ab90c817825
0618 13:59:42.354924 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.354986 (+ 62us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2415512}
I20260618 13:59:42.355602 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 196) took 2421 ms. Trace:
I20260618 13:59:42.355701 29746 rpcz_store.cc:276] 0618 13:59:39.934413 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.939448 (+ 5035us) service_pool.cc:224] Handling call
0618 13:59:42.355459 (+2416011us) ts_tablet_manager.cc:1151] Deleting tablet d4667fe48f0f4eff9c6c8baabdab58d4
0618 13:59:42.355472 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.355584 (+ 112us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2415859}
I20260618 13:59:42.356163 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 199) took 2415 ms. Trace:
I20260618 13:59:42.356242 29746 rpcz_store.cc:276] 0618 13:59:39.941023 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.941073 (+ 50us) service_pool.cc:224] Handling call
0618 13:59:42.356077 (+2415004us) ts_tablet_manager.cc:1151] Deleting tablet dd2385e6752d49429422fc854539e9c6
0618 13:59:42.356088 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.356147 (+ 59us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2412827}
I20260618 13:59:42.356681 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 200) took 2415 ms. Trace:
I20260618 13:59:42.356751 29746 rpcz_store.cc:276] 0618 13:59:39.941626 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.941676 (+ 50us) service_pool.cc:224] Handling call
0618 13:59:42.356608 (+2414932us) ts_tablet_manager.cc:1151] Deleting tablet 94365c44b5e24acab5df60c67203794a
0618 13:59:42.356619 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.356668 (+ 49us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2413242}
I20260618 13:59:42.357074 29746 tablet_replica.cc:333] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:42.357261 29746 raft_consensus.cc:2243] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:42.357417 29746 raft_consensus.cc:2272] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.358063 29746 ts_tablet_manager.cc:1916] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.367554 28798 ts_tablet_manager.cc:1434] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.027s user 0.000s sys 0.001s
I20260618 13:59:42.370880 28798 tablet_bootstrap.cc:492] T ba53ad1988064049a61fcca3a85f8779 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:42.371899 28797 tablet_bootstrap.cc:492] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:42.372044 28797 ts_tablet_manager.cc:1403] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.124s user 0.002s sys 0.000s
I20260618 13:59:42.372892 28797 raft_consensus.cc:359] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.373092 28797 raft_consensus.cc:385] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.373153 28797 raft_consensus.cc:740] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.373360 28797 consensus_queue.cc:260] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.373898 28797 ts_tablet_manager.cc:1434] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:42.374202 28797 tablet_bootstrap.cc:492] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:42.375092 30551 raft_consensus.cc:493] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.375269 30551 raft_consensus.cc:515] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.376044 30551 leader_election.cc:290] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.377024 30623 raft_consensus.cc:493] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.377158 30623 raft_consensus.cc:515] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.377719 30623 leader_election.cc:290] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:42.378528 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f83bb8c3c504d5f9affbaa8be3f1508" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:42.353255 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 8b88ed7cf79c4929a8704f66691cbed4 because the tablet was not found. No further retry: Not found: Tablet not found: 8b88ed7cf79c4929a8704f66691cbed4
I20260618 13:59:42.378933 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:42.378980 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f83bb8c3c504d5f9affbaa8be3f1508" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
W20260618 13:59:42.379020 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 233040a16b0d4b61b977ca57c1afec2d because the tablet was not found. No further retry: Not found: Tablet not found: 233040a16b0d4b61b977ca57c1afec2d
W20260618 13:59:42.379345 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d7a97176f72349cda221baaea61d18ad because the tablet was not found. No further retry: Not found: Tablet not found: d7a97176f72349cda221baaea61d18ad
W20260618 13:59:42.379592 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet e025b64380314546af8a1ab90c817825 because the tablet was not found. No further retry: Not found: Tablet not found: e025b64380314546af8a1ab90c817825
W20260618 13:59:42.379828 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d4667fe48f0f4eff9c6c8baabdab58d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d4667fe48f0f4eff9c6c8baabdab58d4 already in progress: creating tablet
W20260618 13:59:42.380031 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet dd2385e6752d49429422fc854539e9c6 because the tablet was not found. No further retry: Not found: Tablet not found: dd2385e6752d49429422fc854539e9c6
W20260618 13:59:42.380247 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 94365c44b5e24acab5df60c67203794a because the tablet was not found. No further retry: Not found: Tablet not found: 94365c44b5e24acab5df60c67203794a
I20260618 13:59:42.380473 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db62b3a1e37b447c9ea3fbd65c0cf1e7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:42.380599 30577 raft_consensus.cc:493] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.380714 30577 raft_consensus.cc:515] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.381129 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db62b3a1e37b447c9ea3fbd65c0cf1e7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:42.381222 30577 leader_election.cc:290] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.381348 30577 consensus_queue.cc:1048] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:42.382257 28483 leader_election.cc:304] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:42.382834 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c74d3ca66f42ad83ef54d9459b99fd" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:42.383138 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c74d3ca66f42ad83ef54d9459b99fd" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:42.383315 28408 raft_consensus.cc:2468] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:42.384377 28617 leader_election.cc:304] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:42.384721 29303 raft_consensus.cc:2804] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:42.384789 29303 raft_consensus.cc:493] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:42.384851 29303 raft_consensus.cc:3060] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:42.385798 28346 leader_election.cc:343] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:42.385967 28346 leader_election.cc:343] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.386034 28346 leader_election.cc:304] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:42.386631 30152 raft_consensus.cc:2749] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:42.387984 28618 leader_election.cc:343] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.390360 29303 raft_consensus.cc:515] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.391088 29303 leader_election.cc:290] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.391681 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c74d3ca66f42ad83ef54d9459b99fd" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:42.392740 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c74d3ca66f42ad83ef54d9459b99fd" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:42.392907 28407 raft_consensus.cc:3060] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.394641 30623 raft_consensus.cc:493] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.394778 30623 raft_consensus.cc:515] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.395337 30623 leader_election.cc:290] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:42.395958 28407 raft_consensus.cc:2468] T a3c74d3ca66f42ad83ef54d9459b99fd P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:42.396281 30551 raft_consensus.cc:2749] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:42.397416 30551 raft_consensus.cc:493] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.397576 30551 raft_consensus.cc:515] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.398195 30551 leader_election.cc:290] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.391438 28797 tablet_bootstrap.cc:654] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:42.392225 28618 leader_election.cc:343] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.398728 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f7eaa7d218440ca2c35340c5c87814" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:42.398926 28682 raft_consensus.cc:2468] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:42.399401 28484 leader_election.cc:304] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:42.399531 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60e02f51b8624a739fadbf4fafe90cb5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:42.399770 28617 leader_election.cc:304] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:42.400154 30577 raft_consensus.cc:2804] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:42.400264 30577 raft_consensus.cc:697] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:42.400542 30577 consensus_queue.cc:237] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:42.403389 28346 leader_election.cc:343] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.405629 30551 raft_consensus.cc:2804] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:42.405742 30551 raft_consensus.cc:493] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:42.405824 30551 raft_consensus.cc:3060] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.406102 29746 ts_tablet_manager.cc:1929] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:42.406241 29746 log.cc:1199] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/35d8dd2c388e43a0be380aa953ec2c82
I20260618 13:59:42.408686 28799 tablet_bootstrap.cc:492] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:42.408845 28799 ts_tablet_manager.cc:1403] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.079s user 0.016s sys 0.012s
I20260618 13:59:42.409590 28799 raft_consensus.cc:359] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.409758 28799 raft_consensus.cc:385] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.409819 28799 raft_consensus.cc:740] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.410197 30551 raft_consensus.cc:515] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.410941 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:42.411393 28799 consensus_queue.cc:260] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.411473 29023 catalog_manager.cc:5696] T fc295a8e81b64859915a6335f8fb8d8e P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:42.411782 28799 ts_tablet_manager.cc:1434] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:42.412073 28799 tablet_bootstrap.cc:492] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:42.412101 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:42.412400 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:42.412614 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:42.412950 29023 catalog_manager.cc:5696] T b509f940a02545019c526e45e8372c97 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:42.416904 28799 tablet_bootstrap.cc:654] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.417196 29023 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:42.419042 30720 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a950ab7cd66b4cb0b29592aa692c2dd0)
I20260618 13:59:42.419232 30720 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:42.420033 30720 leader_election.cc:290] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a950ab7cd66b4cb0b29592aa692c2dd0 (127.27.100.61:45967), 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083)
I20260618 13:59:42.420771 29048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 655 } } ignore_live_leader: false dest_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" is_pre_election: true
I20260618 13:59:42.423142 29303 consensus_queue.cc:1048] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:42.428216 29566 ts_tablet_manager.cc:1929] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:42.428365 29566 log.cc:1199] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/cd06bb7cffc34412b420d70435fce373
I20260618 13:59:42.428930 29566 ts_tablet_manager.cc:1950] T cd06bb7cffc34412b420d70435fce373 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
W20260618 13:59:42.419209 28483 leader_election.cc:343] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: e5f7eaa7d218440ca2c35340c5c87814
I20260618 13:59:42.429898 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 146) took 9444 ms. Trace:
I20260618 13:59:42.429998 29566 rpcz_store.cc:276] 0618 13:59:32.984948 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:33.013393 (+ 28445us) service_pool.cc:224] Handling call
0618 13:59:42.327735 (+9314342us) ts_tablet_manager.cc:1151] Deleting tablet cd06bb7cffc34412b420d70435fce373
0618 13:59:42.327744 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.400534 (+ 72790us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.428129 (+ 27595us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.429873 (+ 1744us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9314113}
I20260618 13:59:42.430668 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet cd06bb7cffc34412b420d70435fce373 (table default.Table_74fc7c21dc664ff8bfe0d25aee399bdb [id=d224150fd3a340fc96e75c0b96146b11]) successfully deleted
I20260618 13:59:42.431196 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f7eaa7d218440ca2c35340c5c87814" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:42.431388 28682 raft_consensus.cc:3060] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:42.431775 28483 leader_election.cc:343] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: e5f7eaa7d218440ca2c35340c5c87814
I20260618 13:59:42.431926 30551 leader_election.cc:290] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.433943 28682 raft_consensus.cc:2468] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:42.434564 28484 leader_election.cc:304] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:42.435292 30551 raft_consensus.cc:2804] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:42.435524 30551 raft_consensus.cc:697] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:42.436555 30551 consensus_queue.cc:237] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.437688 29566 tablet_replica.cc:333] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:42.437995 29566 raft_consensus.cc:2243] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:42.438385 29566 raft_consensus.cc:2272] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.440016 29566 ts_tablet_manager.cc:1916] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.426821 29746 ts_tablet_manager.cc:1950] T 35d8dd2c388e43a0be380aa953ec2c82 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:42.445744 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 201) took 2503 ms. Trace:
I20260618 13:59:42.445897 29746 rpcz_store.cc:276] 0618 13:59:39.942217 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.942280 (+ 63us) service_pool.cc:224] Handling call
0618 13:59:42.357033 (+2414753us) ts_tablet_manager.cc:1151] Deleting tablet 35d8dd2c388e43a0be380aa953ec2c82
0618 13:59:42.357041 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.402972 (+ 45931us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.406033 (+ 3061us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.445722 (+ 39689us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2413312}
I20260618 13:59:42.446390 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 35d8dd2c388e43a0be380aa953ec2c82 (table default.Table_a8705c5cd49a40049ff1020a01aa7f5b [id=f9ff809bb5da4d048e7520cec3316f6d]) successfully deleted
I20260618 13:59:42.446585 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 202) took 2503 ms. Trace:
I20260618 13:59:42.446692 29746 rpcz_store.cc:276] 0618 13:59:39.942845 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.942897 (+ 52us) service_pool.cc:224] Handling call
0618 13:59:42.446450 (+2503553us) ts_tablet_manager.cc:1151] Deleting tablet f89c9954d48e407c96c294c26644e25c
0618 13:59:42.446464 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.446561 (+ 97us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2502667}
W20260618 13:59:42.447337 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet f89c9954d48e407c96c294c26644e25c because the tablet was not found. No further retry: Not found: Tablet not found: f89c9954d48e407c96c294c26644e25c
I20260618 13:59:42.447783 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 203) took 2503 ms. Trace:
I20260618 13:59:42.447854 29746 rpcz_store.cc:276] 0618 13:59:39.944102 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.944150 (+ 48us) service_pool.cc:224] Handling call
0618 13:59:42.447710 (+2503560us) ts_tablet_manager.cc:1151] Deleting tablet 8fd9fa9675234db2abff694a517c0ea8
0618 13:59:42.447719 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.447768 (+ 49us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2491710}
W20260618 13:59:42.448230 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 8fd9fa9675234db2abff694a517c0ea8 because the tablet was not found. No further retry: Not found: Tablet not found: 8fd9fa9675234db2abff694a517c0ea8
I20260618 13:59:42.448578 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 208) took 2492 ms. Trace:
I20260618 13:59:42.448738 29746 rpcz_store.cc:276] 0618 13:59:39.955918 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.955968 (+ 50us) service_pool.cc:224] Handling call
0618 13:59:42.448506 (+2492538us) ts_tablet_manager.cc:1151] Deleting tablet 8b59255b14df4158afe37cb8890e7415
0618 13:59:42.448514 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.448563 (+ 49us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2492350}
W20260618 13:59:42.449182 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 8b59255b14df4158afe37cb8890e7415 because the tablet was not found. No further retry: Not found: Tablet not found: 8b59255b14df4158afe37cb8890e7415
I20260618 13:59:42.449620 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 204) took 2504 ms. Trace:
I20260618 13:59:42.449759 29746 rpcz_store.cc:276] 0618 13:59:39.944788 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.944841 (+ 53us) service_pool.cc:224] Handling call
0618 13:59:42.449520 (+2504679us) ts_tablet_manager.cc:1151] Deleting tablet 0a51e830e3014726b1d1c2608973f036
0618 13:59:42.449532 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.449598 (+ 66us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2493254}
W20260618 13:59:42.450332 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 0a51e830e3014726b1d1c2608973f036 because the tablet was not found. No further retry: Not found: Tablet not found: 0a51e830e3014726b1d1c2608973f036
I20260618 13:59:42.455188 30551 raft_consensus.cc:493] T fc1968e5ce2e42a5b836c90b0904f158 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.455376 30551 raft_consensus.cc:515] T fc1968e5ce2e42a5b836c90b0904f158 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.456597 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1968e5ce2e42a5b836c90b0904f158" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:42.456907 30551 leader_election.cc:290] T fc1968e5ce2e42a5b836c90b0904f158 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.457049 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1968e5ce2e42a5b836c90b0904f158" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:42.457943 28483 leader_election.cc:304] T fc1968e5ce2e42a5b836c90b0904f158 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:42.459877 28798 tablet_bootstrap.cc:654] T ba53ad1988064049a61fcca3a85f8779 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.461063 30551 raft_consensus.cc:2749] T fc1968e5ce2e42a5b836c90b0904f158 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:42.463042 30623 raft_consensus.cc:493] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.463219 30623 raft_consensus.cc:515] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.464023 30623 leader_election.cc:290] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:42.464891 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7c6c17ba06543219259749d7d38832d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:42.465513 28346 leader_election.cc:343] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.473680 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60e02f51b8624a739fadbf4fafe90cb5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:42.474252 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7c6c17ba06543219259749d7d38832d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:42.474785 28346 leader_election.cc:343] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.474895 28346 leader_election.cc:304] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:42.475312 30623 raft_consensus.cc:2749] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:42.475731 28346 leader_election.cc:343] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.475855 28346 leader_election.cc:304] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:42.476878 30623 raft_consensus.cc:2749] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:42.478595 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 205) took 2527 ms. Trace:
I20260618 13:59:42.478739 29746 rpcz_store.cc:276] 0618 13:59:39.950767 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.950820 (+ 53us) service_pool.cc:224] Handling call
0618 13:59:42.450678 (+2499858us) ts_tablet_manager.cc:1151] Deleting tablet dca54111fea047d9ae0fabe376d89382
0618 13:59:42.450691 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.478568 (+ 27877us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2491841}
I20260618 13:59:42.479341 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 206) took 2527 ms. Trace:
I20260618 13:59:42.479506 29746 rpcz_store.cc:276] 0618 13:59:39.951514 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.951556 (+ 42us) service_pool.cc:224] Handling call
0618 13:59:42.479230 (+2527674us) ts_tablet_manager.cc:1151] Deleting tablet 5c75278db2234b11bf95f1cf5cc76edd
0618 13:59:42.479243 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.479320 (+ 77us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2520254}
I20260618 13:59:42.480082 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 207) took 2525 ms. Trace:
I20260618 13:59:42.480245 29746 rpcz_store.cc:276] 0618 13:59:39.954473 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.954517 (+ 44us) service_pool.cc:224] Handling call
0618 13:59:42.479994 (+2525477us) ts_tablet_manager.cc:1151] Deleting tablet d6871232860b41d489e34d95eb897241
0618 13:59:42.480003 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.480062 (+ 59us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2520902}
I20260618 13:59:42.480847 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 209) took 2522 ms. Trace:
I20260618 13:59:42.481005 29746 rpcz_store.cc:276] 0618 13:59:39.958478 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.958530 (+ 52us) service_pool.cc:224] Handling call
0618 13:59:42.480764 (+2522234us) ts_tablet_manager.cc:1151] Deleting tablet d1baf86497d442dbbbddbc3498bebbee
0618 13:59:42.480772 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.480829 (+ 57us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2521582}
W20260618 13:59:42.481139 28617 consensus_peers.cc:597] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cea4bc13eabc4dd98b4f96723498014c. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:42.481639 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 210) took 2521 ms. Trace:
I20260618 13:59:42.481760 29746 rpcz_store.cc:276] 0618 13:59:39.960295 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.960339 (+ 44us) service_pool.cc:224] Handling call
0618 13:59:42.481561 (+2521222us) ts_tablet_manager.cc:1151] Deleting tablet 41107eb1d9164a9c98060003b4027412
0618 13:59:42.481573 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.481626 (+ 53us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2521072}
I20260618 13:59:42.482545 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 211) took 2521 ms. Trace:
I20260618 13:59:42.482666 29746 rpcz_store.cc:276] 0618 13:59:39.960869 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.960912 (+ 43us) service_pool.cc:224] Handling call
0618 13:59:42.482142 (+2521230us) ts_tablet_manager.cc:1151] Deleting tablet fca76774f119435899188313a33dad98
0618 13:59:42.482150 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.482528 (+ 378us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2521098}
W20260618 13:59:42.482762 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet dca54111fea047d9ae0fabe376d89382 because the tablet was not found. No further retry: Not found: Tablet not found: dca54111fea047d9ae0fabe376d89382
I20260618 13:59:42.483116 29746 tablet_replica.cc:333] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
W20260618 13:59:42.483182 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 5c75278db2234b11bf95f1cf5cc76edd because the tablet was not found. No further retry: Not found: Tablet not found: 5c75278db2234b11bf95f1cf5cc76edd
I20260618 13:59:42.483304 29746 raft_consensus.cc:2243] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:42.483443 29746 raft_consensus.cc:2272] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:42.483412 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d6871232860b41d489e34d95eb897241 because the tablet was not found. No further retry: Not found: Tablet not found: d6871232860b41d489e34d95eb897241
W20260618 13:59:42.483633 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d1baf86497d442dbbbddbc3498bebbee because the tablet was not found. No further retry: Not found: Tablet not found: d1baf86497d442dbbbddbc3498bebbee
W20260618 13:59:42.483846 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 41107eb1d9164a9c98060003b4027412 because the tablet was not found. No further retry: Not found: Tablet not found: 41107eb1d9164a9c98060003b4027412
W20260618 13:59:42.484076 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet fca76774f119435899188313a33dad98 because the tablet was not found. No further retry: Not found: Tablet not found: fca76774f119435899188313a33dad98
W20260618 13:59:42.485827 28071 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111)
I20260618 13:59:42.491161 29746 ts_tablet_manager.cc:1916] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.493835 29566 ts_tablet_manager.cc:1929] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:42.493969 29566 log.cc:1199] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/46bfe8cc2078414d84e65e22ed4ef8a2
I20260618 13:59:42.494380 29566 ts_tablet_manager.cc:1950] T 46bfe8cc2078414d84e65e22ed4ef8a2 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:42.495232 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 161) took 8846 ms. Trace:
I20260618 13:59:42.495330 29566 rpcz_store.cc:276] 0618 13:59:33.648526 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:33.653391 (+ 4865us) service_pool.cc:224] Handling call
0618 13:59:42.437615 (+8784224us) ts_tablet_manager.cc:1151] Deleting tablet 46bfe8cc2078414d84e65e22ed4ef8a2
0618 13:59:42.437633 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.469632 (+ 31999us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.493755 (+ 24123us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.495210 (+ 1455us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8783974}
I20260618 13:59:42.495851 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 46bfe8cc2078414d84e65e22ed4ef8a2 (table default.Table_659a3d495ed44be4bfb2193816362e25 [id=2659803e822d4761a15e0e89644cc472]) successfully deleted
W20260618 13:59:42.496100 28071 leader_election.cc:336] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083): Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111)
I20260618 13:59:42.496253 28071 leader_election.cc:304] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e4d4f4566be42dfbafb7c0e40fb17ff; no voters: 7637285570574a6dbcef1de6f84ecac0, a950ab7cd66b4cb0b29592aa692c2dd0
I20260618 13:59:42.496516 30720 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260618 13:59:42.497964 28799 tablet_bootstrap.cc:492] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:42.498123 28799 ts_tablet_manager.cc:1403] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.086s user 0.013s sys 0.019s
I20260618 13:59:42.499037 28799 raft_consensus.cc:359] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.499249 28799 raft_consensus.cc:385] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.499322 28799 raft_consensus.cc:740] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.499693 28799 consensus_queue.cc:260] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.500612 28799 ts_tablet_manager.cc:1434] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260618 13:59:42.500725 28618 consensus_peers.cc:597] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:42.500869 28799 tablet_bootstrap.cc:492] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:42.502589 28799 tablet_bootstrap.cc:654] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.508544 30577 raft_consensus.cc:493] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.508714 30577 raft_consensus.cc:515] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.509449 30577 leader_election.cc:290] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.510746 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bbb71449d434d54b007d6f0da0796d7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:42.510799 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bbb71449d434d54b007d6f0da0796d7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:42.510931 28408 raft_consensus.cc:2468] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:42.511868 28617 leader_election.cc:304] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:42.512128 29303 raft_consensus.cc:2804] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:42.512197 29303 raft_consensus.cc:493] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:42.512256 29303 raft_consensus.cc:3060] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.513044 28798 tablet_bootstrap.cc:492] T ba53ad1988064049a61fcca3a85f8779 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:42.513164 28798 ts_tablet_manager.cc:1403] T ba53ad1988064049a61fcca3a85f8779 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.142s user 0.003s sys 0.001s
I20260618 13:59:42.513870 28798 raft_consensus.cc:359] T ba53ad1988064049a61fcca3a85f8779 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.514032 28798 raft_consensus.cc:385] T ba53ad1988064049a61fcca3a85f8779 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.514077 28798 raft_consensus.cc:740] T ba53ad1988064049a61fcca3a85f8779 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.514338 28798 consensus_queue.cc:260] T ba53ad1988064049a61fcca3a85f8779 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.515067 28798 ts_tablet_manager.cc:1434] T ba53ad1988064049a61fcca3a85f8779 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260618 13:59:42.516280 28618 leader_election.cc:343] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.516371 28798 tablet_bootstrap.cc:492] T b968a8d4014341ca9c5964263721ac79 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:42.518085 29303 raft_consensus.cc:515] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.519858 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bbb71449d434d54b007d6f0da0796d7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:42.519992 28798 tablet_bootstrap.cc:654] T b968a8d4014341ca9c5964263721ac79 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:42.520825 28618 leader_election.cc:343] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.521900 29303 leader_election.cc:290] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.522698 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bbb71449d434d54b007d6f0da0796d7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:42.522922 28408 raft_consensus.cc:3060] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.526273 29746 ts_tablet_manager.cc:1929] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:42.530552 30623 raft_consensus.cc:493] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.534368 29746 log.cc:1199] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/7aa9f65d8a5c46fdb745ee17af946dce
I20260618 13:59:42.535064 29746 ts_tablet_manager.cc:1950] T 7aa9f65d8a5c46fdb745ee17af946dce P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:42.536029 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 212) took 2573 ms. Trace:
I20260618 13:59:42.536144 29746 rpcz_store.cc:276] 0618 13:59:39.962157 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.962201 (+ 44us) service_pool.cc:224] Handling call
0618 13:59:42.483062 (+2520861us) ts_tablet_manager.cc:1151] Deleting tablet 7aa9f65d8a5c46fdb745ee17af946dce
0618 13:59:42.483073 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.523184 (+ 40111us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.526200 (+ 3016us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.536002 (+ 9802us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2520697}
I20260618 13:59:42.536733 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 7aa9f65d8a5c46fdb745ee17af946dce (table default.Table_c2afd6643b7b49b7992384135658ddfc [id=29178416a03447608c28e809919d9461]) successfully deleted
I20260618 13:59:42.538765 28797 tablet_bootstrap.cc:492] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:42.539011 28797 ts_tablet_manager.cc:1403] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.165s user 0.010s sys 0.009s
I20260618 13:59:42.540001 28797 raft_consensus.cc:359] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.540225 28797 raft_consensus.cc:385] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.540366 28797 raft_consensus.cc:740] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.540673 28797 consensus_queue.cc:260] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.547930 30152 consensus_queue.cc:1048] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:42.550239 28797 ts_tablet_manager.cc:1434] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260618 13:59:42.550640 28797 tablet_bootstrap.cc:492] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:42.552874 29566 tablet_replica.cc:333] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:42.553113 29566 raft_consensus.cc:2243] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:42.553448 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 214) took 2590 ms. Trace:
I20260618 13:59:42.553486 29566 raft_consensus.cc:2272] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.554073 29566 ts_tablet_manager.cc:1916] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.554358 29746 rpcz_store.cc:276] 0618 13:59:39.963440 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.963548 (+ 108us) service_pool.cc:224] Handling call
0618 13:59:42.553343 (+2589795us) ts_tablet_manager.cc:1151] Deleting tablet 316f470ea82f42fca48e4e96d49c9574
0618 13:59:42.553355 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.553428 (+ 73us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2580837}
I20260618 13:59:42.554702 30719 raft_consensus.cc:493] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.554844 30719 raft_consensus.cc:515] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.555739 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c871e7c8f843486da7ba40145c0fed19" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:42.556214 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 215) took 2592 ms. Trace:
W20260618 13:59:42.556236 28618 leader_election.cc:343] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.556304 29746 rpcz_store.cc:276] 0618 13:59:39.964213 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.964259 (+ 46us) service_pool.cc:224] Handling call
0618 13:59:42.556111 (+2591852us) ts_tablet_manager.cc:1151] Deleting tablet aef7c1f111634c98af046e73fa9ffdeb
0618 13:59:42.556124 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.556196 (+ 72us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2583641}
W20260618 13:59:42.556623 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 316f470ea82f42fca48e4e96d49c9574 because the tablet was not found. No further retry: Not found: Tablet not found: 316f470ea82f42fca48e4e96d49c9574
W20260618 13:59:42.556936 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet aef7c1f111634c98af046e73fa9ffdeb because the tablet was not found. No further retry: Not found: Tablet not found: aef7c1f111634c98af046e73fa9ffdeb
I20260618 13:59:42.557541 30719 leader_election.cc:290] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.557854 28797 tablet_bootstrap.cc:654] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.558892 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 216) took 2594 ms. Trace:
I20260618 13:59:42.558979 29746 rpcz_store.cc:276] 0618 13:59:39.964756 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.964803 (+ 47us) service_pool.cc:224] Handling call
0618 13:59:42.558807 (+2594004us) ts_tablet_manager.cc:1151] Deleting tablet eead369de58f40378254697cbbbc6958
0618 13:59:42.558816 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.558876 (+ 60us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2586279}
W20260618 13:59:42.559408 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet eead369de58f40378254697cbbbc6958 because the tablet was not found. No further retry: Not found: Tablet not found: eead369de58f40378254697cbbbc6958
I20260618 13:59:42.560226 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 217) took 2594 ms. Trace:
I20260618 13:59:42.560250 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:42.561407 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:42.562117 29746 rpcz_store.cc:276] 0618 13:59:39.965309 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.965357 (+ 48us) service_pool.cc:224] Handling call
0618 13:59:42.560140 (+2594783us) ts_tablet_manager.cc:1151] Deleting tablet 75824a5d2f7e47838b60e6a2b785c6b2
0618 13:59:42.560149 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.560209 (+ 60us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2587547}
W20260618 13:59:42.564272 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 75824a5d2f7e47838b60e6a2b785c6b2 because the tablet was not found. No further retry: Not found: Tablet not found: 75824a5d2f7e47838b60e6a2b785c6b2
I20260618 13:59:42.530735 30623 raft_consensus.cc:515] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.563732 28798 tablet_bootstrap.cc:492] T b968a8d4014341ca9c5964263721ac79 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:42.565115 30623 leader_election.cc:290] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:42.565694 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daface32f2a94a49a2db0729dfba32c2" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:42.566069 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daface32f2a94a49a2db0729dfba32c2" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:42.566301 28346 leader_election.cc:343] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:42.566538 28346 leader_election.cc:343] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.566666 28346 leader_election.cc:304] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:42.567171 30623 raft_consensus.cc:2749] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:42.565338 28798 ts_tablet_manager.cc:1403] T b968a8d4014341ca9c5964263721ac79 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.050s user 0.004s sys 0.003s
I20260618 13:59:42.568735 28798 raft_consensus.cc:359] T b968a8d4014341ca9c5964263721ac79 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:42.568951 28798 raft_consensus.cc:385] T b968a8d4014341ca9c5964263721ac79 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.569034 28798 raft_consensus.cc:740] T b968a8d4014341ca9c5964263721ac79 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.569324 28798 consensus_queue.cc:260] T b968a8d4014341ca9c5964263721ac79 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:42.571661 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c871e7c8f843486da7ba40145c0fed19" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:42.571848 28411 raft_consensus.cc:2468] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:42.573103 28617 leader_election.cc:304] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:42.573685 28798 ts_tablet_manager.cc:1434] T b968a8d4014341ca9c5964263721ac79 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260618 13:59:42.574774 28798 tablet_bootstrap.cc:492] T fe1139c409e84070a2f496a85545e8d5 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:42.574847 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:42.575114 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 538077e459a44ed78c95f2e881b6e0b0 (table default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307]). Expected version 1 got 0
I20260618 13:59:42.575377 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:42.575400 28408 raft_consensus.cc:2468] T 5bbb71449d434d54b007d6f0da0796d7 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:42.575888 28617 leader_election.cc:304] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:42.576221 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:42.576546 30572 raft_consensus.cc:2804] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:42.576637 30572 raft_consensus.cc:493] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:42.576658 28798 tablet_bootstrap.cc:654] T fe1139c409e84070a2f496a85545e8d5 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.576713 30572 raft_consensus.cc:3060] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.577836 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 218) took 2611 ms. Trace:
I20260618 13:59:42.577926 29746 rpcz_store.cc:276] 0618 13:59:39.966019 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.966068 (+ 49us) service_pool.cc:224] Handling call
0618 13:59:42.577730 (+2611662us) ts_tablet_manager.cc:1151] Deleting tablet 4c2b924a096b479cad31c29bf7c99172
0618 13:59:42.577741 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.577817 (+ 76us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2605094}
I20260618 13:59:42.578209 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
W20260618 13:59:42.578409 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 4c2b924a096b479cad31c29bf7c99172 because the tablet was not found. No further retry: Not found: Tablet not found: 4c2b924a096b479cad31c29bf7c99172
I20260618 13:59:42.578724 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:42.579059 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet be00bfa9bc1a4b83b18bfad05ba1495e (table default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230]). Expected version 2 got 0
I20260618 13:59:42.579581 28797 tablet_bootstrap.cc:492] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:42.579718 28797 ts_tablet_manager.cc:1403] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.000s
I20260618 13:59:42.580085 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_01e1e5741bb54f78996857f6b79fc5d7 [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 2 got 0
I20260618 13:59:42.580473 28797 raft_consensus.cc:359] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.580646 28797 raft_consensus.cc:385] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.580802 30719 raft_consensus.cc:2804] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:42.581033 30719 raft_consensus.cc:697] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:42.581594 30719 consensus_queue.cc:237] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.582756 29566 ts_tablet_manager.cc:1929] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:42.584617 30623 raft_consensus.cc:493] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.584764 30623 raft_consensus.cc:515] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.585350 30623 leader_election.cc:290] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:42.583479 29011 catalog_manager.cc:5696] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:42.586460 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6903cb63c5b04408b7eaecf0c0231c42" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:42.586841 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:42.586150 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6903cb63c5b04408b7eaecf0c0231c42" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:42.587157 28346 leader_election.cc:343] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:42.587416 28346 leader_election.cc:343] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.587522 28346 leader_election.cc:304] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:42.588016 30623 raft_consensus.cc:2749] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:42.588353 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 219) took 2621 ms. Trace:
I20260618 13:59:42.588508 29746 rpcz_store.cc:276] 0618 13:59:39.966684 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.966730 (+ 46us) service_pool.cc:224] Handling call
0618 13:59:42.583571 (+2616841us) ts_tablet_manager.cc:1151] Deleting tablet 962bf304a285412fb2a529db5b994e9e
0618 13:59:42.583581 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.588328 (+ 4747us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2610884}
I20260618 13:59:42.590344 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 220) took 2623 ms. Trace:
I20260618 13:59:42.590796 30572 raft_consensus.cc:515] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.591979 28799 tablet_bootstrap.cc:492] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:42.592120 28799 ts_tablet_manager.cc:1403] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.091s user 0.017s sys 0.020s
I20260618 13:59:42.587589 29566 log.cc:1199] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/9a0691de253b4b008e5d995fd6b876c2
I20260618 13:59:42.592806 28799 raft_consensus.cc:359] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.592906 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c871e7c8f843486da7ba40145c0fed19" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:42.593003 28799 raft_consensus.cc:385] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260618 13:59:42.589999 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 962bf304a285412fb2a529db5b994e9e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 962bf304a285412fb2a529db5b994e9e already in progress: creating tablet
I20260618 13:59:42.593065 28799 raft_consensus.cc:740] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
W20260618 13:59:42.593447 28618 leader_election.cc:343] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.593822 29566 ts_tablet_manager.cc:1950] T 9a0691de253b4b008e5d995fd6b876c2 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:42.594168 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
W20260618 13:59:42.594728 28618 consensus_peers.cc:597] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cea4bc13eabc4dd98b4f96723498014c. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:42.595712 30572 leader_election.cc:290] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.592928 29746 rpcz_store.cc:276] 0618 13:59:39.967249 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.967308 (+ 59us) service_pool.cc:224] Handling call
0618 13:59:42.590253 (+2622945us) ts_tablet_manager.cc:1151] Deleting tablet fddea8fba4da40d7a5780e6750caf5e9
0618 13:59:42.590264 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.590327 (+ 63us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2617535}
I20260618 13:59:42.598111 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 221) took 2630 ms. Trace:
I20260618 13:59:42.598204 29746 rpcz_store.cc:276] 0618 13:59:39.967876 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.967919 (+ 43us) service_pool.cc:224] Handling call
0618 13:59:42.598027 (+2630108us) ts_tablet_manager.cc:1151] Deleting tablet 37a6764a5e024903848561c9935b1a43
0618 13:59:42.598036 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.598094 (+ 58us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2625272}
I20260618 13:59:42.580698 28797 raft_consensus.cc:740] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.593288 28799 consensus_queue.cc:260] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.597393 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c871e7c8f843486da7ba40145c0fed19" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:42.598875 28797 consensus_queue.cc:260] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.598970 28408 raft_consensus.cc:3060] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.599411 28797 ts_tablet_manager.cc:1434] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.020s user 0.002s sys 0.000s
I20260618 13:59:42.599731 28797 tablet_bootstrap.cc:492] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:42.600167 28799 ts_tablet_manager.cc:1434] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
W20260618 13:59:42.597858 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet fddea8fba4da40d7a5780e6750caf5e9 because the tablet was not found. No further retry: Not found: Tablet not found: fddea8fba4da40d7a5780e6750caf5e9
W20260618 13:59:42.600538 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 37a6764a5e024903848561c9935b1a43 because the tablet was not found. No further retry: Not found: Tablet not found: 37a6764a5e024903848561c9935b1a43
I20260618 13:59:42.600751 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 222) took 2632 ms. Trace:
I20260618 13:59:42.600821 29746 rpcz_store.cc:276] 0618 13:59:39.968423 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.968465 (+ 42us) service_pool.cc:224] Handling call
0618 13:59:42.600670 (+2632205us) ts_tablet_manager.cc:1151] Deleting tablet ba0d05ea44e1473fb4b24f61eab096e4
0618 13:59:42.600679 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.600734 (+ 55us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2627876}
W20260618 13:59:42.601233 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet ba0d05ea44e1473fb4b24f61eab096e4 because the tablet was not found. No further retry: Not found: Tablet not found: ba0d05ea44e1473fb4b24f61eab096e4
I20260618 13:59:42.601452 28799 tablet_bootstrap.cc:492] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:42.602108 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 223) took 2633 ms. Trace:
I20260618 13:59:42.602224 29746 rpcz_store.cc:276] 0618 13:59:39.968964 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.969009 (+ 45us) service_pool.cc:224] Handling call
0618 13:59:42.602032 (+2633023us) ts_tablet_manager.cc:1151] Deleting tablet b6dd422c05324c989e29b20ae2afabad
0618 13:59:42.602040 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.602092 (+ 52us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2629197}
I20260618 13:59:42.602371 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 162) took 8677 ms. Trace:
I20260618 13:59:42.602531 29566 rpcz_store.cc:276] 0618 13:59:33.924776 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:33.924976 (+ 200us) service_pool.cc:224] Handling call
0618 13:59:42.552817 (+8627841us) ts_tablet_manager.cc:1151] Deleting tablet 9a0691de253b4b008e5d995fd6b876c2
0618 13:59:42.552829 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.567214 (+ 14385us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.582702 (+ 15488us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.602351 (+ 19649us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8627676}
W20260618 13:59:42.602725 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet b6dd422c05324c989e29b20ae2afabad because the tablet was not found. No further retry: Not found: Tablet not found: b6dd422c05324c989e29b20ae2afabad
I20260618 13:59:42.603585 28408 raft_consensus.cc:2468] T c871e7c8f843486da7ba40145c0fed19 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:42.603976 28617 leader_election.cc:304] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:42.604277 30572 raft_consensus.cc:2804] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:42.604413 30572 raft_consensus.cc:697] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:42.604666 30572 consensus_queue.cc:237] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.605405 28799 tablet_bootstrap.cc:654] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.605990 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet ab100ebbe61a4321b9e76c730e5d46ed (table default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985]). Expected version 1 got 0
I20260618 13:59:42.606189 29746 tablet_replica.cc:333] T c3d523419ff244f085d6eac1a3231973 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:42.606348 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:42.606402 29746 raft_consensus.cc:2243] T c3d523419ff244f085d6eac1a3231973 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:42.606617 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:42.606649 29746 raft_consensus.cc:2272] T c3d523419ff244f085d6eac1a3231973 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.606808 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:42.606835 28546 raft_consensus.cc:3060] T 054a7afb3b644aa49df78302cf8f02c9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.607206 29746 ts_tablet_manager.cc:1916] T c3d523419ff244f085d6eac1a3231973 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.607249 29011 catalog_manager.cc:5696] T b968a8d4014341ca9c5964263721ac79 P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:42.607831 29011 catalog_manager.cc:5696] T eebebb79db644535b54a48e4dee89d59 P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:42.608860 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:42.609077 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:42.609331 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 316110bd2d024c7599fa2933d87e559e (table default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522]). Expected version 1 got 0
I20260618 13:59:42.609781 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:42.614573 28797 tablet_bootstrap.cc:654] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.619118 30572 raft_consensus.cc:493] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.619329 30572 raft_consensus.cc:515] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.620363 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e9a417eecf4d33ac3bd9bd6325f447" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:42.620919 28618 leader_election.cc:343] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.621481 28798 tablet_bootstrap.cc:492] T fe1139c409e84070a2f496a85545e8d5 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:42.621598 28798 ts_tablet_manager.cc:1403] T fe1139c409e84070a2f496a85545e8d5 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.048s user 0.002s sys 0.002s
I20260618 13:59:42.621766 30572 leader_election.cc:290] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.622241 28798 raft_consensus.cc:359] T fe1139c409e84070a2f496a85545e8d5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:42.622397 28798 raft_consensus.cc:385] T fe1139c409e84070a2f496a85545e8d5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.622474 28798 raft_consensus.cc:740] T fe1139c409e84070a2f496a85545e8d5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.622963 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e9a417eecf4d33ac3bd9bd6325f447" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:42.623191 28408 raft_consensus.cc:2468] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:42.623651 28617 leader_election.cc:304] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:42.624101 30572 raft_consensus.cc:2804] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:42.624234 30572 raft_consensus.cc:493] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:42.624341 30572 raft_consensus.cc:3060] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.626966 30572 raft_consensus.cc:515] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.633662 28798 consensus_queue.cc:260] T fe1139c409e84070a2f496a85545e8d5 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:42.633821 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e9a417eecf4d33ac3bd9bd6325f447" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:42.634918 28798 ts_tablet_manager.cc:1434] T fe1139c409e84070a2f496a85545e8d5 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260618 13:59:42.635213 28798 tablet_bootstrap.cc:492] T b509f940a02545019c526e45e8372c97 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
W20260618 13:59:42.635560 28618 leader_election.cc:343] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.610407 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c9ceb3af08fa4618b81cf6f89d7bd0df (table default.Table_21242eba3cb64c7c91082e296439cfe2 [id=10a4f5ef579746b68d8dd37d83f14c3e]). Expected version 1 got 0
I20260618 13:59:42.636693 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 1 got 0
I20260618 13:59:42.637080 30572 leader_election.cc:290] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.637193 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet ba53ad1988064049a61fcca3a85f8779 (table default.Table_b7f097c1abe1410a800ce5aa25771740 [id=6c30b030a60b47e2be48dfe6e9cfdfdb]). Expected version 1 got 0
I20260618 13:59:42.637166 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e9a417eecf4d33ac3bd9bd6325f447" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
W20260618 13:59:42.637256 28346 consensus_peers.cc:597] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:42.637338 28408 raft_consensus.cc:3060] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.637629 29011 catalog_manager.cc:5696] T f1663f9d92e54f1caf3c7137412e495c P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:42.638782 28797 tablet_bootstrap.cc:492] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:42.638917 28797 ts_tablet_manager.cc:1403] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.039s user 0.001s sys 0.002s
I20260618 13:59:42.639700 28797 raft_consensus.cc:359] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.639894 28797 raft_consensus.cc:385] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.639962 28797 raft_consensus.cc:740] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.639979 28408 raft_consensus.cc:2468] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
W20260618 13:59:42.640493 29019 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e196dd3c7fe14524a50152643ddbc2ad consensus state has no leader
I20260618 13:59:42.640480 28617 leader_election.cc:304] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:42.640673 29019 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 131) took 2946 ms. Trace:
I20260618 13:59:42.640928 28797 consensus_queue.cc:260] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.641196 30719 raft_consensus.cc:2804] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:42.640770 29019 rpcz_store.cc:276] 0618 13:59:39.693913 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.868905 (+2174992us) service_pool.cc:224] Handling call
0618 13:59:41.869368 (+ 463us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:41.869601 (+ 233us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:41.869769 (+ 168us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:41.869859 (+ 90us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:41.879907 (+ 10048us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.879913 (+ 6us) write_op.cc:454] Released schema lock
0618 13:59:41.879983 (+ 70us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:42.640637 (+760654us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.870477 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.870573 (+ 96us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.870578 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:41.870580 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:41.870668 (+ 88us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.870680 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.870685 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:41.870742 (+ 57us) tablet.cc:701] Row locks acquired
0618 13:59:41.870744 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:41.870774 (+ 30us) write_op.cc:270] Start()
0618 13:59:41.870802 (+ 28us) write_op.cc:276] Timestamp: P: 1781791181870706 usec, L: 0
0618 13:59:41.870805 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.870975 (+ 170us) log.cc:844] Serialized 405 byte log entry
0618 13:59:41.874881 (+ 3906us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.875058 (+ 177us) write_op.cc:301] APPLY: starting
0618 13:59:41.875078 (+ 20us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.875129 (+ 51us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.875165 (+ 36us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.875211 (+ 46us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.875241 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:41.875248 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:41.875313 (+ 65us) log.cc:844] Serialized 23 byte log entry
0618 13:59:41.875367 (+ 54us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.875385 (+ 18us) write_op.cc:454] Released schema lock
0618 13:59:41.875391 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":123,"apply.run_cpu_time_us":382,"apply.run_wall_time_us":381,"num_ops":1,"prepare.queue_time_us":66,"prepare.run_cpu_time_us":602,"prepare.run_wall_time_us":869,"raft.queue_time_us":103,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":197,"replication_time_us":4042}]]}
I20260618 13:59:42.641299 30719 raft_consensus.cc:697] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:42.641516 30719 consensus_queue.cc:237] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.645045 28458 heartbeater.cc:344] Connected to a master server at 127.27.100.61:45967
I20260618 13:59:42.645218 28458 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:42.645853 28596 heartbeater.cc:344] Connected to a master server at 127.27.100.61:45967
I20260618 13:59:42.645998 28596 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:42.647390 28799 tablet_bootstrap.cc:492] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:42.647522 28799 ts_tablet_manager.cc:1403] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.046s user 0.000s sys 0.009s
I20260618 13:59:42.648547 28799 raft_consensus.cc:359] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.648769 28799 raft_consensus.cc:385] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.648833 28799 raft_consensus.cc:740] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.649073 28799 consensus_queue.cc:260] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.650207 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 9a0691de253b4b008e5d995fd6b876c2 (table default.Table_ab57b782a92747e78e2bfe28480aa0ec [id=28d43fd5660f459a9acfab2fc0dfbdb1]) successfully deleted
I20260618 13:59:42.653106 28797 ts_tablet_manager.cc:1434] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20260618 13:59:42.653409 28797 tablet_bootstrap.cc:492] T e3a3395aa90e426eb48a3920a8c45ba8 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:42.658622 28546 raft_consensus.cc:3060] T 7f83bb8c3c504d5f9affbaa8be3f1508 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.659682 29746 ts_tablet_manager.cc:1929] T c3d523419ff244f085d6eac1a3231973 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:42.659801 29746 log.cc:1199] T c3d523419ff244f085d6eac1a3231973 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/c3d523419ff244f085d6eac1a3231973
I20260618 13:59:42.660180 29746 ts_tablet_manager.cc:1950] T c3d523419ff244f085d6eac1a3231973 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:42.660244 28799 ts_tablet_manager.cc:1434] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.013s user 0.000s sys 0.002s
I20260618 13:59:42.660945 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 224) took 2690 ms. Trace:
I20260618 13:59:42.661039 29746 rpcz_store.cc:276] 0618 13:59:39.970390 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.970461 (+ 71us) service_pool.cc:224] Handling call
0618 13:59:42.606142 (+2635681us) ts_tablet_manager.cc:1151] Deleting tablet c3d523419ff244f085d6eac1a3231973
0618 13:59:42.606151 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.622300 (+ 16149us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.659622 (+ 37322us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.660926 (+ 1304us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2633265}
I20260618 13:59:42.661725 29566 tablet_replica.cc:333] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:42.661933 29566 raft_consensus.cc:2243] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:42.662249 29566 raft_consensus.cc:2272] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.662839 29566 ts_tablet_manager.cc:1916] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:42.664558 28983 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet e196dd3c7fe14524a50152643ddbc2ad (table default.Table_2dfebd62a146446f999f1a7a1bd1af79 [id=02ffed3d852845d6845fb5176f036fce], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e196dd3c7fe14524a50152643ddbc2ad consensus state has no leader
I20260618 13:59:42.667564 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet c3d523419ff244f085d6eac1a3231973 (table default.Table_176a739c87ee45798e2470f8ec00d632 [id=2534d29e39c14366b10492a7c6a92947]) successfully deleted
I20260618 13:59:42.668381 28799 tablet_bootstrap.cc:492] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:42.669413 28797 tablet_bootstrap.cc:654] T e3a3395aa90e426eb48a3920a8c45ba8 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.673549 28798 tablet_bootstrap.cc:654] T b509f940a02545019c526e45e8372c97 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.675665 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 213) took 2712 ms. Trace:
I20260618 13:59:42.676539 29746 rpcz_store.cc:276] 0618 13:59:39.962911 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.962955 (+ 44us) service_pool.cc:224] Handling call
0618 13:59:42.675537 (+2712582us) ts_tablet_manager.cc:1151] Deleting tablet 36b8a46a41e543c58a946b6764618c7d
0618 13:59:42.675560 (+ 23us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.675642 (+ 82us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2702628}
I20260618 13:59:42.678766 28799 tablet_bootstrap.cc:654] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.690819 30577 raft_consensus.cc:493] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.690968 30577 raft_consensus.cc:515] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.691560 30577 leader_election.cc:290] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.695267 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8f124ba79b46e098303b7b2705844d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:42.695458 28408 raft_consensus.cc:2468] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:42.695691 29566 ts_tablet_manager.cc:1929] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:42.695791 29566 log.cc:1199] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/98b42d23832545ae81a7cdff19e4689f
I20260618 13:59:42.696130 29566 ts_tablet_manager.cc:1950] T 98b42d23832545ae81a7cdff19e4689f P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:42.697970 28617 leader_election.cc:304] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:42.698346 29303 raft_consensus.cc:2804] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:42.698433 29303 raft_consensus.cc:493] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:42.698493 29303 raft_consensus.cc:3060] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.700871 29303 raft_consensus.cc:515] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.701429 29303 leader_election.cc:290] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.702265 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8f124ba79b46e098303b7b2705844d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:42.702457 28408 raft_consensus.cc:3060] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.702857 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 163) took 8716 ms. Trace:
I20260618 13:59:42.702932 29566 rpcz_store.cc:276] 0618 13:59:33.985970 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:33.986036 (+ 66us) service_pool.cc:224] Handling call
0618 13:59:42.661670 (+8675634us) ts_tablet_manager.cc:1151] Deleting tablet 98b42d23832545ae81a7cdff19e4689f
0618 13:59:42.661681 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.687715 (+ 26034us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.695634 (+ 7919us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.702834 (+ 7200us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8675382}
I20260618 13:59:42.703357 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 98b42d23832545ae81a7cdff19e4689f (table default.Table_c99584ecc88948a097b39c22d68b3938 [id=7c39b3c432e944bca05f7b2070a79ff9]) successfully deleted
I20260618 13:59:42.703645 29566 tablet_replica.cc:333] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:42.703811 29566 raft_consensus.cc:2243] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:42.703936 29566 raft_consensus.cc:2272] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.704962 28408 raft_consensus.cc:2468] T 9d8f124ba79b46e098303b7b2705844d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:42.705202 30622 raft_consensus.cc:493] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.705313 30622 raft_consensus.cc:515] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.705390 28617 leader_election.cc:304] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:42.705638 29303 raft_consensus.cc:2804] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:42.705736 29303 raft_consensus.cc:697] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:42.705906 30622 leader_election.cc:290] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:42.705956 29303 consensus_queue.cc:237] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.707135 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c200f34abf074684a0314c9024f04caf" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:42.708320 28346 leader_election.cc:343] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.710999 29566 ts_tablet_manager.cc:1916] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.718459 30622 consensus_queue.cc:1048] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:42.720192 30622 raft_consensus.cc:493] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.720317 30622 raft_consensus.cc:515] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.720853 30622 leader_election.cc:290] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:42.722227 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32d411b71d146059d40260e54c283ec" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:42.722942 28346 leader_election.cc:343] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.727655 29566 ts_tablet_manager.cc:1929] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:42.727811 29566 log.cc:1199] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/962bf304a285412fb2a529db5b994e9e
I20260618 13:59:42.728994 29566 ts_tablet_manager.cc:1950] T 962bf304a285412fb2a529db5b994e9e P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:42.729730 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 164) took 8458 ms. Trace:
I20260618 13:59:42.729861 29566 rpcz_store.cc:276] 0618 13:59:34.271268 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.271319 (+ 51us) service_pool.cc:224] Handling call
0618 13:59:42.703602 (+8432283us) ts_tablet_manager.cc:1151] Deleting tablet 962bf304a285412fb2a529db5b994e9e
0618 13:59:42.703611 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.714386 (+ 10775us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.727598 (+ 13212us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.729713 (+ 2115us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8432122}
I20260618 13:59:42.730646 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 962bf304a285412fb2a529db5b994e9e (table default.Table_0fbace1c1aa746a6a2a57c878631218b [id=18b5971926ed4fff9048003a51d3a4fa]) successfully deleted
I20260618 13:59:42.731127 29566 tablet_replica.cc:333] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:42.734478 29566 raft_consensus.cc:2243] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:42.734863 29566 raft_consensus.cc:2272] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.735446 29566 ts_tablet_manager.cc:1916] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.743319 28797 tablet_bootstrap.cc:492] T e3a3395aa90e426eb48a3920a8c45ba8 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:42.743468 28797 ts_tablet_manager.cc:1403] T e3a3395aa90e426eb48a3920a8c45ba8 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.090s user 0.003s sys 0.001s
I20260618 13:59:42.744408 28797 raft_consensus.cc:359] T e3a3395aa90e426eb48a3920a8c45ba8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.744604 28797 raft_consensus.cc:385] T e3a3395aa90e426eb48a3920a8c45ba8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.744668 28797 raft_consensus.cc:740] T e3a3395aa90e426eb48a3920a8c45ba8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.744887 28797 consensus_queue.cc:260] T e3a3395aa90e426eb48a3920a8c45ba8 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.746166 28797 ts_tablet_manager.cc:1434] T e3a3395aa90e426eb48a3920a8c45ba8 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:42.746621 28797 tablet_bootstrap.cc:492] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:42.748529 28797 tablet_bootstrap.cc:654] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.692926 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8f124ba79b46e098303b7b2705844d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:42.694689 28798 tablet_bootstrap.cc:492] T b509f940a02545019c526e45e8372c97 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:42.695518 28540 raft_consensus.cc:3060] T fc1968e5ce2e42a5b836c90b0904f158 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.749733 28798 ts_tablet_manager.cc:1403] T b509f940a02545019c526e45e8372c97 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.115s user 0.001s sys 0.005s
W20260618 13:59:42.749943 28618 leader_election.cc:343] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.750538 28798 raft_consensus.cc:359] T b509f940a02545019c526e45e8372c97 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.750706 29566 ts_tablet_manager.cc:1929] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:42.750737 28798 raft_consensus.cc:385] T b509f940a02545019c526e45e8372c97 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.750782 28798 raft_consensus.cc:740] T b509f940a02545019c526e45e8372c97 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.750798 29566 log.cc:1199] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/35d8dd2c388e43a0be380aa953ec2c82
I20260618 13:59:42.751101 29566 ts_tablet_manager.cc:1950] T 35d8dd2c388e43a0be380aa953ec2c82 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:42.750982 28798 consensus_queue.cc:260] T b509f940a02545019c526e45e8372c97 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.751698 28798 ts_tablet_manager.cc:1434] T b509f940a02545019c526e45e8372c97 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:42.751950 28798 tablet_bootstrap.cc:492] T 3a783590159f4a80964fbbcb4a24eef9 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:42.701951 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8f124ba79b46e098303b7b2705844d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:42.706885 28545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c200f34abf074684a0314c9024f04caf" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:42.723234 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32d411b71d146059d40260e54c283ec" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:42.734505 28518 tablet_service.cc:1511] Processing CreateTablet for tablet dab5f1faf6bf46e2b7d9c3808511ef5a (DEFAULT_TABLE table=default.Table_25d0ebc0b8a4422087089d856a3e1f5d [id=53e714ef65684b84ace254d0feebae0a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:42.753248 28618 leader_election.cc:343] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.753536 28518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dab5f1faf6bf46e2b7d9c3808511ef5a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:42.753643 28346 leader_election.cc:343] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.753769 28346 leader_election.cc:304] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:42.754266 28346 leader_election.cc:343] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.754393 28346 leader_election.cc:304] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:42.754637 30623 raft_consensus.cc:2749] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:42.754781 30623 raft_consensus.cc:2749] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:42.756129 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 168) took 8284 ms. Trace:
I20260618 13:59:42.756210 29566 rpcz_store.cc:276] 0618 13:59:34.471570 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.472113 (+ 543us) service_pool.cc:224] Handling call
0618 13:59:42.731082 (+8258969us) ts_tablet_manager.cc:1151] Deleting tablet 35d8dd2c388e43a0be380aa953ec2c82
0618 13:59:42.731092 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.741680 (+ 10588us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.750652 (+ 8972us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.756111 (+ 5459us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3071,"tablet-delete.queue_time_us":8258726}
I20260618 13:59:42.756832 29566 tablet_replica.cc:333] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:42.756775 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 35d8dd2c388e43a0be380aa953ec2c82 (table default.Table_a8705c5cd49a40049ff1020a01aa7f5b [id=f9ff809bb5da4d048e7520cec3316f6d]) successfully deleted
I20260618 13:59:42.757016 29566 raft_consensus.cc:2243] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:42.757153 29566 raft_consensus.cc:2272] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.757774 29566 ts_tablet_manager.cc:1916] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.765101 28798 tablet_bootstrap.cc:654] T 3a783590159f4a80964fbbcb4a24eef9 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:42.770149 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 36b8a46a41e543c58a946b6764618c7d because the tablet was not found. No further retry: Not found: Tablet not found: 36b8a46a41e543c58a946b6764618c7d
I20260618 13:59:42.770653 29746 tablet_replica.cc:333] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
W20260618 13:59:42.771039 28983 consensus_peers.cc:597] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 -> Peer 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083): Couldn't send request to peer 7637285570574a6dbcef1de6f84ecac0. Status: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260618 13:59:42.771420 29746 raft_consensus.cc:2243] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:42.772921 28660 tablet_service.cc:1511] Processing CreateTablet for tablet dab5f1faf6bf46e2b7d9c3808511ef5a (DEFAULT_TABLE table=default.Table_25d0ebc0b8a4422087089d856a3e1f5d [id=53e714ef65684b84ace254d0feebae0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.773329 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dab5f1faf6bf46e2b7d9c3808511ef5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.775695 29566 ts_tablet_manager.cc:1929] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:42.775789 29566 log.cc:1199] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/97a9f949999446f09359629fe483bf67
I20260618 13:59:42.776072 29566 ts_tablet_manager.cc:1950] T 97a9f949999446f09359629fe483bf67 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:42.780635 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 180) took 7868 ms. Trace:
I20260618 13:59:42.782505 29746 raft_consensus.cc:2272] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.783407 29746 ts_tablet_manager.cc:1916] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.783295 28799 tablet_bootstrap.cc:492] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:42.784132 28799 ts_tablet_manager.cc:1403] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.116s user 0.018s sys 0.007s
I20260618 13:59:42.784161 29566 rpcz_store.cc:276] 0618 13:59:34.912300 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.912360 (+ 60us) service_pool.cc:224] Handling call
0618 13:59:42.756786 (+7844426us) ts_tablet_manager.cc:1151] Deleting tablet 97a9f949999446f09359629fe483bf67
0618 13:59:42.756796 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.765950 (+ 9154us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.775648 (+ 9698us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.780615 (+ 4967us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7844237}
I20260618 13:59:42.785441 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 97a9f949999446f09359629fe483bf67 (table default.Table_9424eec16a284672a0fe7ab3d0852b40 [id=070ea48af78346aa8ff43f2bb369730c]) successfully deleted
I20260618 13:59:42.785871 29566 tablet_replica.cc:333] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:42.786628 29566 raft_consensus.cc:2243] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:42.787559 28383 tablet_service.cc:1511] Processing CreateTablet for tablet dab5f1faf6bf46e2b7d9c3808511ef5a (DEFAULT_TABLE table=default.Table_25d0ebc0b8a4422087089d856a3e1f5d [id=53e714ef65684b84ace254d0feebae0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.787974 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dab5f1faf6bf46e2b7d9c3808511ef5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.788872 28518 tablet_service.cc:1511] Processing CreateTablet for tablet 25463905e343470d941f89a53efbc2d1 (DEFAULT_TABLE table=default.Table_e4086eb879a44971b5f69ef46e8c56c0 [id=1010c45434cb423fa2dc4019b046a3e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.790592 28518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25463905e343470d941f89a53efbc2d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.791112 30552 raft_consensus.cc:493] T b968a8d4014341ca9c5964263721ac79 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.791231 30552 raft_consensus.cc:515] T b968a8d4014341ca9c5964263721ac79 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:42.791774 30552 leader_election.cc:290] T b968a8d4014341ca9c5964263721ac79 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:42.792604 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b968a8d4014341ca9c5964263721ac79" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:42.792306 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b968a8d4014341ca9c5964263721ac79" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:42.798781 29566 raft_consensus.cc:2272] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.798950 28484 leader_election.cc:304] T b968a8d4014341ca9c5964263721ac79 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:42.799355 30552 raft_consensus.cc:2749] T b968a8d4014341ca9c5964263721ac79 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:42.800429 28799 raft_consensus.cc:359] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.801527 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 25463905e343470d941f89a53efbc2d1 (DEFAULT_TABLE table=default.Table_e4086eb879a44971b5f69ef46e8c56c0 [id=1010c45434cb423fa2dc4019b046a3e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.801748 28799 raft_consensus.cc:385] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.801859 28799 raft_consensus.cc:740] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.801927 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25463905e343470d941f89a53efbc2d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.802192 28799 consensus_queue.cc:260] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.803486 30572 raft_consensus.cc:493] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.803617 30572 raft_consensus.cc:515] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.804165 30572 leader_election.cc:290] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.804876 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fea86ca38a741238a59053151087f69" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:42.805781 28617 leader_election.cc:343] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 8fea86ca38a741238a59053151087f69
W20260618 13:59:42.806335 28618 leader_election.cc:343] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.806473 28618 leader_election.cc:304] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:42.806903 30577 raft_consensus.cc:2749] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:42.808470 28528 tablet_service.cc:1511] Processing CreateTablet for tablet e1cf014708ea4c4085b37c6094bf8a20 (DEFAULT_TABLE table=default.Table_41cc80170f054b5c962ef36e7c322fce [id=0029fe4e6b474c0bb1cdd13b109dc3c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.808849 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1cf014708ea4c4085b37c6094bf8a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.809044 28644 tablet_service.cc:1511] Processing CreateTablet for tablet e1cf014708ea4c4085b37c6094bf8a20 (DEFAULT_TABLE table=default.Table_41cc80170f054b5c962ef36e7c322fce [id=0029fe4e6b474c0bb1cdd13b109dc3c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.809362 28644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1cf014708ea4c4085b37c6094bf8a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.808393 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 25463905e343470d941f89a53efbc2d1 (DEFAULT_TABLE table=default.Table_e4086eb879a44971b5f69ef46e8c56c0 [id=1010c45434cb423fa2dc4019b046a3e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.809957 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25463905e343470d941f89a53efbc2d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.812897 28512 tablet_service.cc:1511] Processing CreateTablet for tablet 6827859dd7be4464a9d98ce502df8539 (DEFAULT_TABLE table=default.Table_7a3559b2cbe044d3b44cf3961c7d2d51 [id=92ed1d0bd76d4556abb3dc7dcf0f5659]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.813212 28512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6827859dd7be4464a9d98ce502df8539. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.814677 28517 tablet_service.cc:1511] Processing CreateTablet for tablet 134a43259d5b429fbe47d0555d9942f3 (DEFAULT_TABLE table=default.Table_34210bd1a3164a189c219c1ed1baeb97 [id=92ec79713ab440ce9d4ca38df71ec124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.815016 28517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 134a43259d5b429fbe47d0555d9942f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.814996 28375 tablet_service.cc:1511] Processing CreateTablet for tablet e1cf014708ea4c4085b37c6094bf8a20 (DEFAULT_TABLE table=default.Table_41cc80170f054b5c962ef36e7c322fce [id=0029fe4e6b474c0bb1cdd13b109dc3c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.815330 28375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1cf014708ea4c4085b37c6094bf8a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.815732 28390 tablet_service.cc:1511] Processing CreateTablet for tablet 66ada05b01b2427da182998a557529ca (DEFAULT_TABLE table=default.Table_982b749682bb4ee8ba1b7a9c4c897f08 [id=7a03f8699e9242309ac380f2b6f02abd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.818674 28384 tablet_service.cc:1511] Processing CreateTablet for tablet 134a43259d5b429fbe47d0555d9942f3 (DEFAULT_TABLE table=default.Table_34210bd1a3164a189c219c1ed1baeb97 [id=92ec79713ab440ce9d4ca38df71ec124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.818740 29566 ts_tablet_manager.cc:1916] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.819044 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 66ada05b01b2427da182998a557529ca (DEFAULT_TABLE table=default.Table_982b749682bb4ee8ba1b7a9c4c897f08 [id=7a03f8699e9242309ac380f2b6f02abd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.819343 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66ada05b01b2427da182998a557529ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.820636 30622 consensus_queue.cc:1048] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:42.822696 28646 tablet_service.cc:1511] Processing CreateTablet for tablet 66ada05b01b2427da182998a557529ca (DEFAULT_TABLE table=default.Table_982b749682bb4ee8ba1b7a9c4c897f08 [id=7a03f8699e9242309ac380f2b6f02abd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.818693 28645 tablet_service.cc:1511] Processing CreateTablet for tablet 134a43259d5b429fbe47d0555d9942f3 (DEFAULT_TABLE table=default.Table_34210bd1a3164a189c219c1ed1baeb97 [id=92ec79713ab440ce9d4ca38df71ec124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.823716 28798 tablet_bootstrap.cc:492] T 3a783590159f4a80964fbbcb4a24eef9 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:42.823853 28798 ts_tablet_manager.cc:1403] T 3a783590159f4a80964fbbcb4a24eef9 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.072s user 0.002s sys 0.004s
I20260618 13:59:42.824638 28798 raft_consensus.cc:359] T 3a783590159f4a80964fbbcb4a24eef9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.824921 28798 raft_consensus.cc:385] T 3a783590159f4a80964fbbcb4a24eef9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.825001 28798 raft_consensus.cc:740] T 3a783590159f4a80964fbbcb4a24eef9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.825207 28798 consensus_queue.cc:260] T 3a783590159f4a80964fbbcb4a24eef9 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.825747 28798 ts_tablet_manager.cc:1434] T 3a783590159f4a80964fbbcb4a24eef9 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:42.826050 28798 tablet_bootstrap.cc:492] T fc295a8e81b64859915a6335f8fb8d8e P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:42.827526 28798 tablet_bootstrap.cc:654] T fc295a8e81b64859915a6335f8fb8d8e P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.831951 28646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66ada05b01b2427da182998a557529ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.833525 29746 ts_tablet_manager.cc:1929] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:42.833652 29746 log.cc:1199] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/f3ba9186b3ff47d5a2b158fef619686e
I20260618 13:59:42.833966 29746 ts_tablet_manager.cc:1950] T f3ba9186b3ff47d5a2b158fef619686e P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:42.834672 28645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 134a43259d5b429fbe47d0555d9942f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.843492 30552 raft_consensus.cc:493] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.843647 30552 raft_consensus.cc:515] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:42.844319 30552 leader_election.cc:290] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:42.844458 28797 tablet_bootstrap.cc:492] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:42.844599 28797 ts_tablet_manager.cc:1403] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.098s user 0.002s sys 0.000s
I20260618 13:59:42.845295 28797 raft_consensus.cc:359] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.845507 28797 raft_consensus.cc:385] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.845595 28797 raft_consensus.cc:740] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.845834 28797 consensus_queue.cc:260] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.846328 28797 ts_tablet_manager.cc:1434] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:42.851418 28517 tablet_service.cc:1511] Processing CreateTablet for tablet 00b602a473a94c05a617fb9478300c74 (DEFAULT_TABLE table=default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.851805 28517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00b602a473a94c05a617fb9478300c74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.853107 29019 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:42.853329 29019 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
W20260618 13:59:42.855129 28483 leader_election.cc:343] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 97a9f949999446f09359629fe483bf67
I20260618 13:59:42.855968 28377 tablet_service.cc:1511] Processing CreateTablet for tablet 6827859dd7be4464a9d98ce502df8539 (DEFAULT_TABLE table=default.Table_7a3559b2cbe044d3b44cf3961c7d2d51 [id=92ed1d0bd76d4556abb3dc7dcf0f5659]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.856433 28798 tablet_bootstrap.cc:492] T fc295a8e81b64859915a6335f8fb8d8e P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:42.856433 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 4da3d6fa934047c09b3815c4bef511e1 (DEFAULT_TABLE table=default.Table_38ca98ca393f4d549e1184885e55e64b [id=8e1dd5cf875a4b38a12372d5064b4a22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.856647 28798 ts_tablet_manager.cc:1403] T fc295a8e81b64859915a6335f8fb8d8e P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.000s
I20260618 13:59:42.856775 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4da3d6fa934047c09b3815c4bef511e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.856386 28797 tablet_bootstrap.cc:492] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:42.858158 28798 raft_consensus.cc:359] T fc295a8e81b64859915a6335f8fb8d8e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:42.858614 28346 consensus_peers.cc:597] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:42.858440 28798 raft_consensus.cc:385] T fc295a8e81b64859915a6335f8fb8d8e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.858712 28798 raft_consensus.cc:740] T fc295a8e81b64859915a6335f8fb8d8e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.858754 28386 tablet_service.cc:1511] Processing CreateTablet for tablet 00b602a473a94c05a617fb9478300c74 (DEFAULT_TABLE table=default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.859098 28798 consensus_queue.cc:260] T fc295a8e81b64859915a6335f8fb8d8e P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.860128 28798 ts_tablet_manager.cc:1434] T fc295a8e81b64859915a6335f8fb8d8e P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:42.861272 28798 tablet_bootstrap.cc:492] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
W20260618 13:59:42.862021 28484 leader_election.cc:343] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 97a9f949999446f09359629fe483bf67
I20260618 13:59:42.862125 28484 leader_election.cc:304] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:42.849063 29566 ts_tablet_manager.cc:1929] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:42.850482 28644 tablet_service.cc:1511] Processing CreateTablet for tablet 6827859dd7be4464a9d98ce502df8539 (DEFAULT_TABLE table=default.Table_7a3559b2cbe044d3b44cf3961c7d2d51 [id=92ed1d0bd76d4556abb3dc7dcf0f5659]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.854660 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 00b602a473a94c05a617fb9478300c74 (DEFAULT_TABLE table=default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.865477 30552 raft_consensus.cc:2749] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:42.865403 28644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6827859dd7be4464a9d98ce502df8539. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.866190 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00b602a473a94c05a617fb9478300c74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.873764 28798 tablet_bootstrap.cc:654] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.876546 28540 raft_consensus.cc:3060] T b968a8d4014341ca9c5964263721ac79 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.878232 28797 tablet_bootstrap.cc:654] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.890621 28799 ts_tablet_manager.cc:1434] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.106s user 0.001s sys 0.001s
I20260618 13:59:42.891211 28799 tablet_bootstrap.cc:492] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:42.892086 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 225) took 2918 ms. Trace:
I20260618 13:59:42.892287 29746 rpcz_store.cc:276] 0618 13:59:39.973276 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.973315 (+ 39us) service_pool.cc:224] Handling call
0618 13:59:42.770586 (+2797271us) ts_tablet_manager.cc:1151] Deleting tablet f3ba9186b3ff47d5a2b158fef619686e
0618 13:59:42.770599 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.830676 (+ 60077us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.833472 (+ 2796us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.892026 (+ 58554us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2797149}
I20260618 13:59:42.893070 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet f3ba9186b3ff47d5a2b158fef619686e (table default.Table_3ef2ff183a6044e2a9e069a4eac133b8 [id=60e91b2138bf41a5a8dee65c1874678a]) successfully deleted
I20260618 13:59:42.893275 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 226) took 2919 ms. Trace:
I20260618 13:59:42.893541 29746 rpcz_store.cc:276] 0618 13:59:39.973894 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.973938 (+ 44us) service_pool.cc:224] Handling call
0618 13:59:42.893078 (+2919140us) ts_tablet_manager.cc:1151] Deleting tablet d9c00003300246ada2a356e3e1cf684b
0618 13:59:42.893091 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.893226 (+ 135us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2918488}
W20260618 13:59:42.894163 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d9c00003300246ada2a356e3e1cf684b because the tablet was not found. No further retry: Not found: Tablet not found: d9c00003300246ada2a356e3e1cf684b
I20260618 13:59:42.894243 29746 tablet_replica.cc:333] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:42.894590 29746 raft_consensus.cc:2243] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:42.895360 29746 raft_consensus.cc:2272] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.865062 29566 log.cc:1199] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/9c17e76ae9b44207998a43e3576908c4
I20260618 13:59:42.896116 30152 raft_consensus.cc:493] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.896255 30152 raft_consensus.cc:515] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.896895 30152 leader_election.cc:290] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:42.897380 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3a3395aa90e426eb48a3920a8c45ba8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:42.897804 28346 leader_election.cc:343] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.897934 29566 ts_tablet_manager.cc:1950] T 9c17e76ae9b44207998a43e3576908c4 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:42.898053 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3a3395aa90e426eb48a3920a8c45ba8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:42.898222 28682 raft_consensus.cc:2468] T e3a3395aa90e426eb48a3920a8c45ba8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:42.898628 28346 leader_election.cc:304] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:42.898872 30262 raft_consensus.cc:2804] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:42.898970 30262 raft_consensus.cc:493] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:42.899045 30262 raft_consensus.cc:3060] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.900624 28797 tablet_bootstrap.cc:492] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:42.900887 28797 ts_tablet_manager.cc:1403] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.045s user 0.001s sys 0.005s
I20260618 13:59:42.901549 30262 raft_consensus.cc:515] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.902156 30262 leader_election.cc:290] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:42.902292 28797 raft_consensus.cc:359] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.902565 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3a3395aa90e426eb48a3920a8c45ba8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:42.902647 28797 raft_consensus.cc:385] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.902870 28797 raft_consensus.cc:740] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.903146 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3a3395aa90e426eb48a3920a8c45ba8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:42.903294 28682 raft_consensus.cc:3060] T e3a3395aa90e426eb48a3920a8c45ba8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.903496 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 182) took 7929 ms. Trace:
I20260618 13:59:42.903429 28797 consensus_queue.cc:260] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.903656 29566 rpcz_store.cc:276] 0618 13:59:34.973508 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.973553 (+ 45us) service_pool.cc:224] Handling call
0618 13:59:42.785821 (+7812268us) ts_tablet_manager.cc:1151] Deleting tablet 9c17e76ae9b44207998a43e3576908c4
0618 13:59:42.785831 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:42.821575 (+ 35744us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.848997 (+ 27422us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:42.903444 (+ 54447us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7812028}
I20260618 13:59:42.904985 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 9c17e76ae9b44207998a43e3576908c4 (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]) successfully deleted
I20260618 13:59:42.906141 28682 raft_consensus.cc:2468] T e3a3395aa90e426eb48a3920a8c45ba8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
W20260618 13:59:42.906617 28346 leader_election.cc:343] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.906800 28346 leader_election.cc:304] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:42.907042 30262 raft_consensus.cc:2804] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:42.907248 30262 raft_consensus.cc:697] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:42.907521 30262 consensus_queue.cc:237] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.907699 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 4da3d6fa934047c09b3815c4bef511e1 (DEFAULT_TABLE table=default.Table_38ca98ca393f4d549e1184885e55e64b [id=8e1dd5cf875a4b38a12372d5064b4a22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.908239 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 4da3d6fa934047c09b3815c4bef511e1 (DEFAULT_TABLE table=default.Table_38ca98ca393f4d549e1184885e55e64b [id=8e1dd5cf875a4b38a12372d5064b4a22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.909297 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4da3d6fa934047c09b3815c4bef511e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.909556 29019 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:42.912663 28797 ts_tablet_manager.cc:1434] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260618 13:59:42.915220 28797 tablet_bootstrap.cc:492] T a662bb37fb7d4df49e61e524808b67a8 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:42.918167 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 43853b8aba3444c3a21f6416f224e0d6 (DEFAULT_TABLE table=default.Table_11c4074a927242a480bca5668deafc83 [id=4c4606b6b5054c3ebfecfeaf6959c3b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.918658 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43853b8aba3444c3a21f6416f224e0d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.922611 28798 tablet_bootstrap.cc:492] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:42.923899 28798 ts_tablet_manager.cc:1403] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.063s user 0.005s sys 0.002s
I20260618 13:59:42.924589 30572 raft_consensus.cc:493] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.924722 30572 raft_consensus.cc:515] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.925316 30572 leader_election.cc:290] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.925508 28546 raft_consensus.cc:3060] T 3a783590159f4a80964fbbcb4a24eef9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.927927 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c4a31a4f202473d97b17b5b5dfdb615" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:42.928493 28618 leader_election.cc:343] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.928805 28644 tablet_service.cc:1511] Processing CreateTablet for tablet 43853b8aba3444c3a21f6416f224e0d6 (DEFAULT_TABLE table=default.Table_11c4074a927242a480bca5668deafc83 [id=4c4606b6b5054c3ebfecfeaf6959c3b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.929545 28644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43853b8aba3444c3a21f6416f224e0d6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:42.929597 28617 leader_election.cc:343] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 6c4a31a4f202473d97b17b5b5dfdb615
I20260618 13:59:42.929817 28617 leader_election.cc:304] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:42.930459 30577 raft_consensus.cc:2749] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:42.927901 29566 tablet_replica.cc:333] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:42.932082 30719 consensus_queue.cc:1048] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:42.934352 30577 consensus_queue.cc:1048] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:42.938338 28798 raft_consensus.cc:359] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.939922 29019 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c9ceb3af08fa4618b81cf6f89d7bd0df (table default.Table_21242eba3cb64c7c91082e296439cfe2 [id=10a4f5ef579746b68d8dd37d83f14c3e]). Expected version 1 got 0
I20260618 13:59:42.940462 29019 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:42.940639 28798 raft_consensus.cc:385] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:42.941044 28798 raft_consensus.cc:740] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:42.942881 29566 raft_consensus.cc:2243] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:42.943282 29566 raft_consensus.cc:2272] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:42.942652 28798 consensus_queue.cc:260] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:42.944342 28618 consensus_peers.cc:597] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f3ba9186b3ff47d5a2b158fef619686e. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:42.944504 29566 ts_tablet_manager.cc:1916] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.950831 29746 ts_tablet_manager.cc:1916] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:42.951398 28798 ts_tablet_manager.cc:1434] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.027s user 0.000s sys 0.003s
I20260618 13:59:42.955277 28374 tablet_service.cc:1511] Processing CreateTablet for tablet 43853b8aba3444c3a21f6416f224e0d6 (DEFAULT_TABLE table=default.Table_11c4074a927242a480bca5668deafc83 [id=4c4606b6b5054c3ebfecfeaf6959c3b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.957316 28798 tablet_bootstrap.cc:492] T f1663f9d92e54f1caf3c7137412e495c P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:42.958544 28390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66ada05b01b2427da182998a557529ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.961745 30552 raft_consensus.cc:493] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.961912 30552 raft_consensus.cc:515] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.962096 28374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43853b8aba3444c3a21f6416f224e0d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.962455 28377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6827859dd7be4464a9d98ce502df8539. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.962615 30552 leader_election.cc:290] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:42.966301 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4da3d6fa934047c09b3815c4bef511e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.966652 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00b602a473a94c05a617fb9478300c74. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:42.967167 28483 leader_election.cc:343] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 5e25027f423144b088dad955c7282921
I20260618 13:59:42.967473 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 134a43259d5b429fbe47d0555d9942f3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:42.971408 28484 leader_election.cc:343] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 5e25027f423144b088dad955c7282921
I20260618 13:59:42.971532 28484 leader_election.cc:304] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:42.971740 30552 raft_consensus.cc:2749] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:42.971997 28799 tablet_bootstrap.cc:654] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.973071 28798 tablet_bootstrap.cc:654] T f1663f9d92e54f1caf3c7137412e495c P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:42.973603 28618 consensus_peers.cc:597] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:42.978753 28546 raft_consensus.cc:3060] T b509f940a02545019c526e45e8372c97 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.979161 30152 raft_consensus.cc:493] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.979293 30152 raft_consensus.cc:515] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.979387 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 3f9ec5f31f6c4d37b7c3ec585ed3211b (DEFAULT_TABLE table=default.Table_7ae70108b1714e5ca374cc653bed7b33 [id=0cf9d7859b7d4d95bf769070c43003d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.979733 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f9ec5f31f6c4d37b7c3ec585ed3211b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.979893 30152 leader_election.cc:290] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:42.979868 28797 tablet_bootstrap.cc:654] T a662bb37fb7d4df49e61e524808b67a8 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:42.980108 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038ea0e9c0b04db8b7ec56b065e3cc51" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:42.980588 28540 raft_consensus.cc:3060] T ba53ad1988064049a61fcca3a85f8779 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:42.979161 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 3f9ec5f31f6c4d37b7c3ec585ed3211b (DEFAULT_TABLE table=default.Table_7ae70108b1714e5ca374cc653bed7b33 [id=0cf9d7859b7d4d95bf769070c43003d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:42.987721 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f9ec5f31f6c4d37b7c3ec585ed3211b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:42.982467 30552 raft_consensus.cc:493] T fe1139c409e84070a2f496a85545e8d5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.989015 30552 raft_consensus.cc:515] T fe1139c409e84070a2f496a85545e8d5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:42.989701 30552 leader_election.cc:290] T fe1139c409e84070a2f496a85545e8d5 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:42.989796 29019 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:42.990290 29019 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:42.991003 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe1139c409e84070a2f496a85545e8d5" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
W20260618 13:59:42.993197 28346 leader_election.cc:343] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.994385 30152 raft_consensus.cc:493] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:42.994624 30152 raft_consensus.cc:515] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:42.995289 30152 leader_election.cc:290] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:42.996079 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43224c621a4774870d44f0237b514a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:42.997048 28346 leader_election.cc:343] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:42.980618 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038ea0e9c0b04db8b7ec56b065e3cc51" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:42.983731 28644 tablet_service.cc:1511] Processing CreateTablet for tablet 3f9ec5f31f6c4d37b7c3ec585ed3211b (DEFAULT_TABLE table=default.Table_7ae70108b1714e5ca374cc653bed7b33 [id=0cf9d7859b7d4d95bf769070c43003d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:43.004483 28644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f9ec5f31f6c4d37b7c3ec585ed3211b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:43.004621 28346 leader_election.cc:343] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.004722 28346 leader_election.cc:304] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.004966 30152 raft_consensus.cc:2749] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:42.990638 29019 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:43.005635 29019 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:43.005873 28799 tablet_bootstrap.cc:492] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:43.005906 29019 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet e196dd3c7fe14524a50152643ddbc2ad (table default.Table_2dfebd62a146446f999f1a7a1bd1af79 [id=02ffed3d852845d6845fb5176f036fce]). Expected version 1 got 0
I20260618 13:59:43.006026 28799 ts_tablet_manager.cc:1403] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.115s user 0.003s sys 0.000s
I20260618 13:59:42.995601 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe1139c409e84070a2f496a85545e8d5" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:42.997044 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43224c621a4774870d44f0237b514a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:43.003705 28618 consensus_peers.cc:597] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:43.007021 28484 leader_election.cc:304] T fe1139c409e84070a2f496a85545e8d5 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.007169 28799 raft_consensus.cc:359] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.007404 28799 raft_consensus.cc:385] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.007434 30552 raft_consensus.cc:2749] T fe1139c409e84070a2f496a85545e8d5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.007511 28799 raft_consensus.cc:740] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.007915 28799 consensus_queue.cc:260] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.008855 28799 ts_tablet_manager.cc:1434] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:43.009359 28799 tablet_bootstrap.cc:492] T e44ad8cb18de42e795da5f229f49cdfe P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:43.009804 29019 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
W20260618 13:59:43.012728 28346 leader_election.cc:343] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.012876 28346 leader_election.cc:304] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.012917 29019 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_01e1e5741bb54f78996857f6b79fc5d7 [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 2 got 0
I20260618 13:59:43.013111 29019 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:43.013119 30152 raft_consensus.cc:2749] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.013337 29019 catalog_manager.cc:5696] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:43.013718 29019 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:43.016113 29746 ts_tablet_manager.cc:1929] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:43.016427 28799 tablet_bootstrap.cc:654] T e44ad8cb18de42e795da5f229f49cdfe P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.018486 28797 tablet_bootstrap.cc:492] T a662bb37fb7d4df49e61e524808b67a8 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:43.018702 28797 ts_tablet_manager.cc:1403] T a662bb37fb7d4df49e61e524808b67a8 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.104s user 0.004s sys 0.000s
I20260618 13:59:43.019516 30152 raft_consensus.cc:493] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.019490 28797 raft_consensus.cc:359] T a662bb37fb7d4df49e61e524808b67a8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.019665 28797 raft_consensus.cc:385] T a662bb37fb7d4df49e61e524808b67a8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.019636 30152 raft_consensus.cc:515] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.019719 28797 raft_consensus.cc:740] T a662bb37fb7d4df49e61e524808b67a8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.019961 28797 consensus_queue.cc:260] T a662bb37fb7d4df49e61e524808b67a8 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.020145 30152 leader_election.cc:290] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.020530 29566 ts_tablet_manager.cc:1929] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:43.020627 29566 log.cc:1199] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/e5f7eaa7d218440ca2c35340c5c87814
I20260618 13:59:43.021164 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2128dbec33db453ba51070767a15b21e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:43.021780 28346 leader_election.cc:343] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.024703 29566 ts_tablet_manager.cc:1950] T e5f7eaa7d218440ca2c35340c5c87814 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:43.025358 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 184) took 8051 ms. Trace:
I20260618 13:59:43.025426 29566 rpcz_store.cc:276] 0618 13:59:34.974384 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.974474 (+ 90us) service_pool.cc:224] Handling call
0618 13:59:42.926626 (+7952152us) ts_tablet_manager.cc:1151] Deleting tablet e5f7eaa7d218440ca2c35340c5c87814
0618 13:59:42.926637 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.007781 (+ 81144us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.020480 (+ 12699us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.025340 (+ 4860us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7949544}
I20260618 13:59:43.025892 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet e5f7eaa7d218440ca2c35340c5c87814 (table default.Table_9a78338c3e414b0abea1281082202f57 [id=5697846ccd9c4c659755a0f9b8a66b72]) successfully deleted
I20260618 13:59:43.015066 29019 catalog_manager.cc:5696] T d4539f767c054e9e8ebc1f3f56108e7e P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:43.026532 29746 log.cc:1199] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/c4cfca9e76e641a7bc0b58b9ee240f36
I20260618 13:59:43.026401 29019 catalog_manager.cc:5696] T 5b9f9649587b4c31951204b8bebe7ed8 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:43.026904 29746 ts_tablet_manager.cc:1950] T c4cfca9e76e641a7bc0b58b9ee240f36 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:43.027109 29019 catalog_manager.cc:5696] T 8f7b66652f464893ad812d27e883a9a4 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:43.027570 29019 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:43.016429 30774 tablet.cc:1724] T ba53ad1988064049a61fcca3a85f8779 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:43.027791 29019 catalog_manager.cc:5696] T 9559c5f2c9e94e858093dd2f0865c865 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:43.020999 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2128dbec33db453ba51070767a15b21e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:43.028338 29019 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:43.028550 29019 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:43.029500 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 227) took 3055 ms. Trace:
I20260618 13:59:43.029469 29019 catalog_manager.cc:5696] T f695a7e6a925470787a708c365e6b86b P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:43.029583 29746 rpcz_store.cc:276] 0618 13:59:39.974404 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.979028 (+ 4624us) service_pool.cc:224] Handling call
0618 13:59:42.894160 (+2915132us) ts_tablet_manager.cc:1151] Deleting tablet c4cfca9e76e641a7bc0b58b9ee240f36
0618 13:59:42.894171 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.010363 (+116192us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.016059 (+ 5696us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.029481 (+ 13422us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2914949}
I20260618 13:59:43.029929 29019 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:43.030074 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet c4cfca9e76e641a7bc0b58b9ee240f36 (table default.Table_0e9f574416d84a0f826fa9a1fcc56210 [id=098fa24bb71e4fe1aecc33aea1275ef0]) successfully deleted
I20260618 13:59:43.030560 29023 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a] alter complete (version 1)
I20260618 13:59:43.030750 29025 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 129) took 5300 ms. Trace:
I20260618 13:59:43.030856 29025 rpcz_store.cc:276] 0618 13:59:37.730298 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.613641 (+3883343us) service_pool.cc:224] Handling call
0618 13:59:41.614591 (+ 950us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:41.614837 (+ 246us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:41.615034 (+ 197us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:41.615121 (+ 87us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:41.639821 (+ 24700us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:43.030681 (+1390860us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.615649 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.615761 (+ 112us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.615767 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:41.615769 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:41.615890 (+ 121us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.615902 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.615906 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:41.615984 (+ 78us) tablet.cc:701] Row locks acquired
0618 13:59:41.615987 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:41.616033 (+ 46us) write_op.cc:270] Start()
0618 13:59:41.616065 (+ 32us) write_op.cc:276] Timestamp: P: 1781791181615988 usec, L: 0
0618 13:59:41.616070 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.616244 (+ 174us) log.cc:844] Serialized 405 byte log entry
0618 13:59:41.631791 (+ 15547us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.635043 (+ 3252us) write_op.cc:301] APPLY: starting
0618 13:59:41.635056 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.635088 (+ 32us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.635122 (+ 34us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.635164 (+ 42us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.635189 (+ 25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:41.635196 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:41.635256 (+ 60us) log.cc:844] Serialized 23 byte log entry
0618 13:59:41.635322 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.635339 (+ 17us) write_op.cc:454] Released schema lock
0618 13:59:41.635344 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:41.635408 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.635412 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3197,"apply.run_cpu_time_us":361,"apply.run_wall_time_us":361,"num_ops":1,"prepare.queue_time_us":50,"prepare.run_cpu_time_us":692,"prepare.run_wall_time_us":692,"raft.queue_time_us":380,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":7,"replication_time_us":15682}]]}
W20260618 13:59:43.034668 29020 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8af2bec820b842b4bf6b57482007bd6a consensus state has no leader
I20260618 13:59:43.034844 29020 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 128) took 5308 ms. Trace:
I20260618 13:59:43.034924 29020 rpcz_store.cc:276] 0618 13:59:37.726837 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.523787 (+3796950us) service_pool.cc:224] Handling call
0618 13:59:41.524944 (+ 1157us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:41.526393 (+ 1449us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:41.526612 (+ 219us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:41.526686 (+ 74us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:41.607343 (+ 80657us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:43.034790 (+1427447us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.527855 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.527983 (+ 128us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.527990 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:41.527994 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:41.528114 (+ 120us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.528131 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.528137 (+ 6us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:41.528216 (+ 79us) tablet.cc:701] Row locks acquired
0618 13:59:41.528219 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:41.528256 (+ 37us) write_op.cc:270] Start()
0618 13:59:41.528294 (+ 38us) write_op.cc:276] Timestamp: P: 1781791181528221 usec, L: 0
0618 13:59:41.528299 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.528493 (+ 194us) log.cc:844] Serialized 405 byte log entry
0618 13:59:41.577517 (+ 49024us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.606800 (+ 29283us) write_op.cc:301] APPLY: starting
0618 13:59:41.606821 (+ 21us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.606868 (+ 47us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.606904 (+ 36us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.606956 (+ 52us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.606987 (+ 31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:41.606995 (+ 8us) write_op.cc:312] APPLY: finished
0618 13:59:41.607102 (+ 107us) log.cc:844] Serialized 23 byte log entry
0618 13:59:41.607178 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.607197 (+ 19us) write_op.cc:454] Released schema lock
0618 13:59:41.607203 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:41.607280 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.607284 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":29208,"apply.run_cpu_time_us":478,"apply.run_wall_time_us":480,"num_ops":1,"prepare.queue_time_us":202,"prepare.run_cpu_time_us":750,"prepare.run_wall_time_us":750,"raft.queue_time_us":68,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":49181}]]}
I20260618 13:59:43.035794 28797 ts_tablet_manager.cc:1434] T a662bb37fb7d4df49e61e524808b67a8 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.017s user 0.002s sys 0.000s
I20260618 13:59:43.037993 28798 tablet_bootstrap.cc:492] T f1663f9d92e54f1caf3c7137412e495c P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:43.038131 28798 ts_tablet_manager.cc:1403] T f1663f9d92e54f1caf3c7137412e495c P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.081s user 0.002s sys 0.003s
I20260618 13:59:43.038211 29026 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3
I20260618 13:59:43.038782 29026 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 125) took 5424 ms. Trace:
I20260618 13:59:43.038923 28798 raft_consensus.cc:359] T f1663f9d92e54f1caf3c7137412e495c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.039100 28798 raft_consensus.cc:385] T f1663f9d92e54f1caf3c7137412e495c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.039150 28798 raft_consensus.cc:740] T f1663f9d92e54f1caf3c7137412e495c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.038897 29026 rpcz_store.cc:276] 0618 13:59:37.614335 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.516604 (+3902269us) service_pool.cc:224] Handling call
0618 13:59:41.517184 (+ 580us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:41.517324 (+ 140us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:41.517385 (+ 61us) catalog_manager.cc:2849] Locking tablets
0618 13:59:41.823538 (+306153us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:41.830317 (+ 6779us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:41.830439 (+ 122us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:43.038143 (+1207704us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:43.038748 (+ 605us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.824511 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.824657 (+ 146us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.824663 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:41.824665 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:41.824796 (+ 131us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.824821 (+ 25us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.824826 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:41.824956 (+ 130us) tablet.cc:701] Row locks acquired
0618 13:59:41.824961 (+ 5us) write_op.cc:260] PREPARE: finished
0618 13:59:41.825001 (+ 40us) write_op.cc:270] Start()
0618 13:59:41.825040 (+ 39us) write_op.cc:276] Timestamp: P: 1781791181824934 usec, L: 0
0618 13:59:41.825044 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.825239 (+ 195us) log.cc:844] Serialized 728 byte log entry
0618 13:59:41.829381 (+ 4142us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.829542 (+ 161us) write_op.cc:301] APPLY: starting
0618 13:59:41.829561 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.829612 (+ 51us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.829639 (+ 27us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.829705 (+ 66us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.829740 (+ 35us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:41.829747 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:41.829835 (+ 88us) log.cc:844] Serialized 29 byte log entry
0618 13:59:41.829898 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.829918 (+ 20us) write_op.cc:454] Released schema lock
0618 13:59:41.829927 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:41.830006 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.830013 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":108,"apply.run_cpu_time_us":451,"apply.run_wall_time_us":451,"num_ops":2,"prepare.queue_time_us":56,"prepare.run_cpu_time_us":854,"prepare.run_wall_time_us":855,"raft.queue_time_us":156,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":265,"replication_time_us":4306}]]}
I20260618 13:59:43.039389 28798 consensus_queue.cc:260] T f1663f9d92e54f1caf3c7137412e495c P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.039740 29026 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_a4350f6ab3f646c881c841735019f707"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:43.040042 28798 ts_tablet_manager.cc:1434] T f1663f9d92e54f1caf3c7137412e495c P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260618 13:59:43.040287 29026 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4350f6ab3f646c881c841735019f707 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:43.040328 28798 tablet_bootstrap.cc:492] T 5b9f9649587b4c31951204b8bebe7ed8 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:43.041396 29024 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet fea86899d2c047d997cebe80551e83c9
I20260618 13:59:43.041843 29024 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 127) took 5321 ms. Trace:
I20260618 13:59:43.042045 28798 tablet_bootstrap.cc:654] T 5b9f9649587b4c31951204b8bebe7ed8 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.041963 29024 rpcz_store.cc:276] 0618 13:59:37.720374 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.521838 (+3801464us) service_pool.cc:224] Handling call
0618 13:59:41.522366 (+ 528us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:41.529712 (+ 7346us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:41.529765 (+ 53us) catalog_manager.cc:2849] Locking tablets
0618 13:59:41.529926 (+ 161us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:41.630989 (+101063us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:41.631066 (+ 77us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:43.041328 (+1410262us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:43.041804 (+ 476us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.560785 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.560946 (+ 161us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.560951 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:41.560953 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:41.561374 (+ 421us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.561390 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.561397 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:41.561523 (+ 126us) tablet.cc:701] Row locks acquired
0618 13:59:41.561525 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:41.561562 (+ 37us) write_op.cc:270] Start()
0618 13:59:41.561596 (+ 34us) write_op.cc:276] Timestamp: P: 1781791181561523 usec, L: 0
0618 13:59:41.561600 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.561799 (+ 199us) log.cc:844] Serialized 728 byte log entry
0618 13:59:41.622043 (+ 60244us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.630501 (+ 8458us) write_op.cc:301] APPLY: starting
0618 13:59:41.630523 (+ 22us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.630562 (+ 39us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.630588 (+ 26us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.630649 (+ 61us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.630678 (+ 29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:41.630685 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:41.630751 (+ 66us) log.cc:844] Serialized 29 byte log entry
0618 13:59:41.630813 (+ 62us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.630830 (+ 17us) write_op.cc:454] Released schema lock
0618 13:59:41.630837 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:41.630907 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.630911 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":8384,"apply.run_cpu_time_us":401,"apply.run_wall_time_us":401,"num_ops":2,"prepare.queue_time_us":812,"prepare.run_cpu_time_us":888,"prepare.run_wall_time_us":1143,"raft.queue_time_us":437,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":7,"replication_time_us":60408}]]}
I20260618 13:59:43.042719 29024 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_689e7dd93b784dc5bbded16bd6cba279" } modify_external_catalogs: true
I20260618 13:59:43.042897 29024 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_689e7dd93b784dc5bbded16bd6cba279" } modify_external_catalogs: true
W20260618 13:59:43.042987 28346 leader_election.cc:343] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.043116 28346 leader_election.cc:304] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.043445 29017 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 2128dbec33db453ba51070767a15b21e
I20260618 13:59:43.043759 30152 raft_consensus.cc:2749] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.043908 29017 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 126) took 5343 ms. Trace:
I20260618 13:59:43.044000 29017 rpcz_store.cc:276] 0618 13:59:37.700678 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.519152 (+3818474us) service_pool.cc:224] Handling call
0618 13:59:41.519617 (+ 465us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:41.519785 (+ 168us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:41.519838 (+ 53us) catalog_manager.cc:2849] Locking tablets
0618 13:59:41.801698 (+281860us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:41.811187 (+ 9489us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.811194 (+ 7us) write_op.cc:454] Released schema lock
0618 13:59:41.811280 (+ 86us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:41.811372 (+ 92us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:43.043408 (+1232036us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:43.043838 (+ 430us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.802648 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.802833 (+ 185us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.802842 (+ 9us) write_op.cc:435] Acquired schema lock
0618 13:59:41.802847 (+ 5us) tablet.cc:662] Decoding operations
0618 13:59:41.803005 (+ 158us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.803023 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.803031 (+ 8us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:41.803174 (+ 143us) tablet.cc:701] Row locks acquired
0618 13:59:41.803178 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:41.803220 (+ 42us) write_op.cc:270] Start()
0618 13:59:41.803265 (+ 45us) write_op.cc:276] Timestamp: P: 1781791181803155 usec, L: 0
0618 13:59:41.803271 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.803503 (+ 232us) log.cc:844] Serialized 734 byte log entry
0618 13:59:41.807029 (+ 3526us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.807227 (+ 198us) write_op.cc:301] APPLY: starting
0618 13:59:41.807248 (+ 21us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.807309 (+ 61us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.807359 (+ 50us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.807466 (+ 107us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.807514 (+ 48us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:41.807526 (+ 12us) write_op.cc:312] APPLY: finished
0618 13:59:41.807625 (+ 99us) log.cc:844] Serialized 29 byte log entry
0618 13:59:41.807699 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.807727 (+ 28us) write_op.cc:454] Released schema lock
0618 13:59:41.807737 (+ 10us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":112,"apply.run_cpu_time_us":575,"apply.run_wall_time_us":574,"num_ops":2,"prepare.queue_time_us":206,"prepare.run_cpu_time_us":999,"prepare.run_wall_time_us":999,"raft.queue_time_us":815,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":203,"replication_time_us":3718}]]}
I20260618 13:59:43.044677 29017 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_a7356ad0d2d04b9788d4d980093d0df4" } new_table_name: "default.Table_c61ea74d5c6d4a718da2ab92f4a52397" modify_external_catalogs: true
I20260618 13:59:43.058149 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:43.058660 28540 raft_consensus.cc:3060] T d2d54a901e9f4d5c89700979ce0618bf P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.063768 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:43.063886 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:43.064100 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 2128dbec33db453ba51070767a15b21e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:43.064131 28645 tablet_service.cc:1558] Processing DeleteTablet for tablet fea86899d2c047d997cebe80551e83c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:43.064311 28646 tablet_service.cc:1558] Processing DeleteTablet for tablet 2128dbec33db453ba51070767a15b21e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:43.064469 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet fea86899d2c047d997cebe80551e83c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:43.065510 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet fea86899d2c047d997cebe80551e83c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:43.067148 29303 raft_consensus.cc:493] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 97859748af2c41879ba2a668dda999d2)
I20260618 13:59:43.067286 29303 raft_consensus.cc:515] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.067891 29303 leader_election.cc:290] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:43.069060 28617 leader_election.cc:343] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 3dd0eaa64271489d92aa380df877f257
I20260618 13:59:43.069895 28798 tablet_bootstrap.cc:492] T 5b9f9649587b4c31951204b8bebe7ed8 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
W20260618 13:59:43.070042 28618 leader_election.cc:343] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Not found: Tablet not found: 3dd0eaa64271489d92aa380df877f257
I20260618 13:59:43.070155 28618 leader_election.cc:304] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:43.070382 28798 ts_tablet_manager.cc:1403] T 5b9f9649587b4c31951204b8bebe7ed8 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.030s user 0.007s sys 0.001s
I20260618 13:59:43.070662 30719 raft_consensus.cc:2749] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:43.077435 28798 raft_consensus.cc:359] T 5b9f9649587b4c31951204b8bebe7ed8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.078214 28798 raft_consensus.cc:385] T 5b9f9649587b4c31951204b8bebe7ed8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.078397 28798 raft_consensus.cc:740] T 5b9f9649587b4c31951204b8bebe7ed8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.078861 28798 consensus_queue.cc:260] T 5b9f9649587b4c31951204b8bebe7ed8 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.080024 28798 ts_tablet_manager.cc:1434] T 5b9f9649587b4c31951204b8bebe7ed8 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20260618 13:59:43.081100 28798 tablet_bootstrap.cc:492] T 55ae29c3b7ed44cfa659130745037aa6 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:43.083359 29011 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_4339ff084ae34b5eaad45284187a4d43 [id=b3649400476d486f8ec75e38db1480c3] alter complete (version 2)
I20260618 13:59:43.084993 28798 tablet_bootstrap.cc:654] T 55ae29c3b7ed44cfa659130745037aa6 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.085289 28377 tablet_service.cc:1558] Processing DeleteTablet for tablet 2128dbec33db453ba51070767a15b21e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:43.091032 29026 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 132) took 2436 ms. Trace:
I20260618 13:59:43.091190 29026 rpcz_store.cc:276] 0618 13:59:40.654877 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:43.039545 (+2384668us) service_pool.cc:224] Handling call
0618 13:59:43.040374 (+ 829us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:43.040818 (+ 444us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:43.088004 (+ 47186us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:43.090843 (+ 2839us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:43.090971 (+ 128us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:43.041881 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:43.042012 (+ 131us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:43.042017 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:43.042019 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:43.042084 (+ 65us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:43.042091 (+ 7us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:43.042095 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:43.042153 (+ 58us) tablet.cc:701] Row locks acquired
0618 13:59:43.042155 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:43.042181 (+ 26us) write_op.cc:270] Start()
0618 13:59:43.042204 (+ 23us) write_op.cc:276] Timestamp: P: 1781791183042156 usec, L: 0
0618 13:59:43.042207 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:43.042342 (+ 135us) log.cc:844] Serialized 433 byte log entry
0618 13:59:43.081309 (+ 38967us) op_driver.cc:464] REPLICATION: finished
0618 13:59:43.082736 (+ 1427us) write_op.cc:301] APPLY: starting
0618 13:59:43.082747 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:43.082787 (+ 40us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:43.082823 (+ 36us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:43.082937 (+ 114us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:43.082969 (+ 32us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:43.082975 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:43.083035 (+ 60us) log.cc:844] Serialized 29 byte log entry
0618 13:59:43.083083 (+ 48us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:43.083097 (+ 14us) write_op.cc:454] Released schema lock
0618 13:59:43.083104 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1390,"apply.run_cpu_time_us":434,"apply.run_wall_time_us":434,"num_ops":2,"prepare.queue_time_us":594,"prepare.run_cpu_time_us":532,"prepare.run_wall_time_us":532,"raft.queue_time_us":117,"raft.run_cpu_time_us":2,"raft.run_wall_time_us":3,"replication_time_us":39085,"threads_started":1}]]}
I20260618 13:59:43.093747 28798 tablet_bootstrap.cc:492] T 55ae29c3b7ed44cfa659130745037aa6 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:43.093920 28798 ts_tablet_manager.cc:1403] T 55ae29c3b7ed44cfa659130745037aa6 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260618 13:59:43.095463 28798 raft_consensus.cc:359] T 55ae29c3b7ed44cfa659130745037aa6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.095652 28798 raft_consensus.cc:385] T 55ae29c3b7ed44cfa659130745037aa6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.095721 28798 raft_consensus.cc:740] T 55ae29c3b7ed44cfa659130745037aa6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.098281 28798 consensus_queue.cc:260] T 55ae29c3b7ed44cfa659130745037aa6 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.099886 28798 ts_tablet_manager.cc:1434] T 55ae29c3b7ed44cfa659130745037aa6 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260618 13:59:43.102078 28798 tablet_bootstrap.cc:492] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:43.105175 28798 tablet_bootstrap.cc:654] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.105963 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 228) took 3124 ms. Trace:
I20260618 13:59:43.106057 29746 rpcz_store.cc:276] 0618 13:59:39.981580 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:39.981623 (+ 43us) service_pool.cc:224] Handling call
0618 13:59:43.105868 (+3124245us) ts_tablet_manager.cc:1151] Deleting tablet 516421cd843b44769efa2b555697c065
0618 13:59:43.105877 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.105946 (+ 69us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3124101}
W20260618 13:59:43.106527 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 516421cd843b44769efa2b555697c065 because the tablet was not found. No further retry: Not found: Tablet not found: 516421cd843b44769efa2b555697c065
I20260618 13:59:43.094370 30777 tablet_replica.cc:333] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:43.107573 30777 raft_consensus.cc:2243] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:43.108414 28797 tablet_bootstrap.cc:492] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:43.109719 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 229) took 2936 ms. Trace:
I20260618 13:59:43.109783 29746 rpcz_store.cc:276] 0618 13:59:40.173660 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.173725 (+ 65us) service_pool.cc:224] Handling call
0618 13:59:43.109639 (+2935914us) ts_tablet_manager.cc:1151] Deleting tablet 2f5a334d18ba4c06bb358fa86d8af80d
0618 13:59:43.109647 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.109705 (+ 58us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2935689}
W20260618 13:59:43.110219 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 2f5a334d18ba4c06bb358fa86d8af80d because the tablet was not found. No further retry: Not found: Tablet not found: 2f5a334d18ba4c06bb358fa86d8af80d
I20260618 13:59:43.110635 28797 tablet_bootstrap.cc:654] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.111580 29566 tablet_replica.cc:333] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:43.111752 29566 raft_consensus.cc:2243] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:43.112053 29566 pending_rounds.cc:70] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257: Trying to abort 1 pending ops.
I20260618 13:59:43.112108 29566 pending_rounds.cc:77] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298216664312455168 op_type: NO_OP noop_request { }
I20260618 13:59:43.112217 29566 raft_consensus.cc:2889] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:43.112285 29566 raft_consensus.cc:2272] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:43.112728 29566 ts_tablet_manager.cc:1916] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:43.109411 28799 tablet_bootstrap.cc:492] T e44ad8cb18de42e795da5f229f49cdfe P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:43.114557 28799 ts_tablet_manager.cc:1403] T e44ad8cb18de42e795da5f229f49cdfe P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.105s user 0.001s sys 0.002s
I20260618 13:59:43.115398 28799 raft_consensus.cc:359] T e44ad8cb18de42e795da5f229f49cdfe P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.115602 28799 raft_consensus.cc:385] T e44ad8cb18de42e795da5f229f49cdfe P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.115681 28799 raft_consensus.cc:740] T e44ad8cb18de42e795da5f229f49cdfe P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.115903 28799 consensus_queue.cc:260] T e44ad8cb18de42e795da5f229f49cdfe P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.116704 28799 ts_tablet_manager.cc:1434] T e44ad8cb18de42e795da5f229f49cdfe P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:43.116910 30777 raft_consensus.cc:2272] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:43.117444 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 236) took 2929 ms. Trace:
I20260618 13:59:43.118327 30777 ts_tablet_manager.cc:1916] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:43.118603 29746 rpcz_store.cc:276] 0618 13:59:40.188361 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.188422 (+ 61us) service_pool.cc:224] Handling call
0618 13:59:43.117358 (+2928936us) ts_tablet_manager.cc:1151] Deleting tablet fe750fc5d998448caeab0a6c808dfba7
0618 13:59:43.117368 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.117428 (+ 60us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2928698}
I20260618 13:59:43.121098 29566 ts_tablet_manager.cc:1929] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:43.121196 29566 log.cc:1199] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/ca2ee746c5024cfc9b31c470f9616e7d
I20260618 13:59:43.121493 29566 ts_tablet_manager.cc:1950] T ca2ee746c5024cfc9b31c470f9616e7d P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:43.122129 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 185) took 8143 ms. Trace:
I20260618 13:59:43.122207 29566 rpcz_store.cc:276] 0618 13:59:34.978990 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.979038 (+ 48us) service_pool.cc:224] Handling call
0618 13:59:43.031404 (+8052366us) ts_tablet_manager.cc:1151] Deleting tablet ca2ee746c5024cfc9b31c470f9616e7d
0618 13:59:43.090654 (+ 59250us) spinlock_profiling.cc:243] Waited 59 ms on lock 0x61800000ed18. stack: 00007f39a472f496 00007f39a472f250 00007f39a375a3ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007f39a4724f2d 00007f39a4725f9e 00007f39b85da1ad 00007f39b85591d3 00007f39b46e2e96 00007f39b46e4229 00007f39b4717bb4 00007f39b46f85d0 00007f39b46f835d 0000000000aa681c
0618 13:59:43.111519 (+ 20865us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.118472 (+ 6953us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.121057 (+ 2585us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.122111 (+ 1054us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135613696,"tablet-delete.queue_time_us":8052177}
I20260618 13:59:43.123302 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet ca2ee746c5024cfc9b31c470f9616e7d (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]) successfully deleted
I20260618 13:59:43.123597 29026 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_e98f724cea1e4b7c8acd2a357fb845a0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:43.124212 29026 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e98f724cea1e4b7c8acd2a357fb845a0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:43.125005 29566 tablet_replica.cc:333] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:43.125208 29566 raft_consensus.cc:2243] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:43.125386 29566 raft_consensus.cc:2272] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:43.128839 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet fe750fc5d998448caeab0a6c808dfba7 because the tablet was not found. No further retry: Not found: Tablet not found: fe750fc5d998448caeab0a6c808dfba7
I20260618 13:59:43.129285 30787 tablet.cc:1724] T 9ead386562eb4fd787c7aee776c2194f P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260618 13:59:43.131165 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 230) took 2947 ms. Trace:
I20260618 13:59:43.131254 29746 rpcz_store.cc:276] 0618 13:59:40.183639 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.183706 (+ 67us) service_pool.cc:224] Handling call
0618 13:59:43.131072 (+2947366us) ts_tablet_manager.cc:1151] Deleting tablet c84189a4b1194c87bbdf81f6e0ed56e4
0618 13:59:43.131090 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.131148 (+ 58us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2942278}
W20260618 13:59:43.131688 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet c84189a4b1194c87bbdf81f6e0ed56e4 because the tablet was not found. No further retry: Not found: Tablet not found: c84189a4b1194c87bbdf81f6e0ed56e4
I20260618 13:59:43.119874 30152 raft_consensus.cc:493] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.131932 30152 raft_consensus.cc:515] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.132501 30152 leader_election.cc:290] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.133104 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a9a5f630654465fb416f53af1424e9b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:43.133610 28346 leader_election.cc:343] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.134030 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 231) took 2949 ms. Trace:
I20260618 13:59:43.133968 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a9a5f630654465fb416f53af1424e9b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:43.134109 29746 rpcz_store.cc:276] 0618 13:59:40.184512 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.184576 (+ 64us) service_pool.cc:224] Handling call
0618 13:59:43.133938 (+2949362us) ts_tablet_manager.cc:1151] Deleting tablet 0fae928833e44d2ba860d6184d4c0142
0618 13:59:43.133947 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.134014 (+ 67us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2945002}
W20260618 13:59:43.134518 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 0fae928833e44d2ba860d6184d4c0142 because the tablet was not found. No further retry: Not found: Tablet not found: 0fae928833e44d2ba860d6184d4c0142
W20260618 13:59:43.134512 28346 leader_election.cc:343] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.134620 28346 leader_election.cc:304] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.134883 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 232) took 2949 ms. Trace:
I20260618 13:59:43.134979 30152 raft_consensus.cc:2749] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.135114 29746 rpcz_store.cc:276] 0618 13:59:40.185306 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.185366 (+ 60us) service_pool.cc:224] Handling call
0618 13:59:43.134812 (+2949446us) ts_tablet_manager.cc:1151] Deleting tablet aa8cf927bed9437086efd8e0f21ba26a
0618 13:59:43.134820 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.134868 (+ 48us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2945765}
W20260618 13:59:43.135497 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet aa8cf927bed9437086efd8e0f21ba26a because the tablet was not found. No further retry: Not found: Tablet not found: aa8cf927bed9437086efd8e0f21ba26a
I20260618 13:59:43.135836 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 237) took 2946 ms. Trace:
I20260618 13:59:43.135946 29746 rpcz_store.cc:276] 0618 13:59:40.189687 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.189743 (+ 56us) service_pool.cc:224] Handling call
0618 13:59:43.135761 (+2946018us) ts_tablet_manager.cc:1151] Deleting tablet bce7bf74b6d24d0bb43808b8a62cf1b7
0618 13:59:43.135770 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.135822 (+ 52us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2945841}
W20260618 13:59:43.136471 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet bce7bf74b6d24d0bb43808b8a62cf1b7 because the tablet was not found. No further retry: Not found: Tablet not found: bce7bf74b6d24d0bb43808b8a62cf1b7
I20260618 13:59:43.137782 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 238) took 2947 ms. Trace:
I20260618 13:59:43.138510 29746 rpcz_store.cc:276] 0618 13:59:40.190408 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.190503 (+ 95us) service_pool.cc:224] Handling call
0618 13:59:43.137708 (+2947205us) ts_tablet_manager.cc:1151] Deleting tablet b0285c75c632457383e81805ab036aaf
0618 13:59:43.137716 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.137767 (+ 51us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2946977}
I20260618 13:59:43.139562 28799 tablet_bootstrap.cc:492] T c9382a86a2f645b4857aa04e459209d9 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:43.140553 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet b0285c75c632457383e81805ab036aaf because the tablet was not found. No further retry: Not found: Tablet not found: b0285c75c632457383e81805ab036aaf
I20260618 13:59:43.141223 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 239) took 2949 ms. Trace:
I20260618 13:59:43.141283 28799 tablet_bootstrap.cc:654] T c9382a86a2f645b4857aa04e459209d9 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.141335 29746 rpcz_store.cc:276] 0618 13:59:40.191292 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.191347 (+ 55us) service_pool.cc:224] Handling call
0618 13:59:43.141145 (+2949798us) ts_tablet_manager.cc:1151] Deleting tablet b16f7d3c7837498e996bba1818110642
0618 13:59:43.141154 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.141206 (+ 52us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2949574}
W20260618 13:59:43.141736 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet b16f7d3c7837498e996bba1818110642 because the tablet was not found. No further retry: Not found: Tablet not found: b16f7d3c7837498e996bba1818110642
I20260618 13:59:43.142661 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 240) took 2950 ms. Trace:
I20260618 13:59:43.142776 29746 rpcz_store.cc:276] 0618 13:59:40.192181 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.192237 (+ 56us) service_pool.cc:224] Handling call
0618 13:59:43.142579 (+2950342us) ts_tablet_manager.cc:1151] Deleting tablet bc0adedecee4405e86b0a37b9e62e352
0618 13:59:43.142591 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.142647 (+ 56us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2950129}
I20260618 13:59:43.142946 29566 ts_tablet_manager.cc:1916] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:43.144728 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet bc0adedecee4405e86b0a37b9e62e352 because the tablet was not found. No further retry: Not found: Tablet not found: bc0adedecee4405e86b0a37b9e62e352
I20260618 13:59:43.145072 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 241) took 2952 ms. Trace:
I20260618 13:59:43.146610 29746 rpcz_store.cc:276] 0618 13:59:40.193036 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.193092 (+ 56us) service_pool.cc:224] Handling call
0618 13:59:43.144995 (+2951903us) ts_tablet_manager.cc:1151] Deleting tablet 751cd9bd961b4f56a588ed02a05f0212
0618 13:59:43.145004 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.145057 (+ 53us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2951712}
W20260618 13:59:43.147127 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 751cd9bd961b4f56a588ed02a05f0212 because the tablet was not found. No further retry: Not found: Tablet not found: 751cd9bd961b4f56a588ed02a05f0212
I20260618 13:59:43.148241 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 233) took 2962 ms. Trace:
I20260618 13:59:43.148357 29746 rpcz_store.cc:276] 0618 13:59:40.186017 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.186077 (+ 60us) service_pool.cc:224] Handling call
0618 13:59:43.148167 (+2962090us) ts_tablet_manager.cc:1151] Deleting tablet 4d6b01b897134d5bbd6a7d93b82ddb5e
0618 13:59:43.148176 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.148225 (+ 49us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2953478}
W20260618 13:59:43.149734 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e because the tablet was not found. No further retry: Not found: Tablet not found: 4d6b01b897134d5bbd6a7d93b82ddb5e
I20260618 13:59:43.150066 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 243) took 2955 ms. Trace:
I20260618 13:59:43.150549 29746 rpcz_store.cc:276] 0618 13:59:40.194575 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.194772 (+ 197us) service_pool.cc:224] Handling call
0618 13:59:43.149980 (+2955208us) ts_tablet_manager.cc:1151] Deleting tablet d667ea5f029f46d89fcb3be03069b97b
0618 13:59:43.150000 (+ 20us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.150051 (+ 51us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2955027}
W20260618 13:59:43.152312 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet d667ea5f029f46d89fcb3be03069b97b because the tablet was not found. No further retry: Not found: Tablet not found: d667ea5f029f46d89fcb3be03069b97b
I20260618 13:59:43.149750 30152 raft_consensus.cc:493] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.153445 30152 raft_consensus.cc:515] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.154323 30152 leader_election.cc:290] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.155002 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbfebfca436461ab63dfe469ae14b4c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:43.155601 28346 leader_election.cc:343] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.155915 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbfebfca436461ab63dfe469ae14b4c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:43.156392 28346 leader_election.cc:343] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.156487 28346 leader_election.cc:304] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.156842 30152 raft_consensus.cc:2749] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.157757 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 234) took 2970 ms. Trace:
I20260618 13:59:43.157842 29746 rpcz_store.cc:276] 0618 13:59:40.186820 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.186879 (+ 59us) service_pool.cc:224] Handling call
0618 13:59:43.157677 (+2970798us) ts_tablet_manager.cc:1151] Deleting tablet b51d0851efb34eaa9e9e4be6b292ef62
0618 13:59:43.157686 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.157740 (+ 54us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2962636}
W20260618 13:59:43.158252 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet b51d0851efb34eaa9e9e4be6b292ef62 because the tablet was not found. No further retry: Not found: Tablet not found: b51d0851efb34eaa9e9e4be6b292ef62
I20260618 13:59:43.161932 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 235) took 2974 ms. Trace:
I20260618 13:59:43.162072 29746 rpcz_store.cc:276] 0618 13:59:40.187585 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.187644 (+ 59us) service_pool.cc:224] Handling call
0618 13:59:43.161860 (+2974216us) ts_tablet_manager.cc:1151] Deleting tablet 092a9aba9c6e48f5be9cf5d55ba93678
0618 13:59:43.161868 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.161917 (+ 49us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2966753}
W20260618 13:59:43.162535 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 092a9aba9c6e48f5be9cf5d55ba93678 because the tablet was not found. No further retry: Not found: Tablet not found: 092a9aba9c6e48f5be9cf5d55ba93678
I20260618 13:59:43.162878 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 242) took 2969 ms. Trace:
I20260618 13:59:43.163663 28799 tablet_bootstrap.cc:492] T c9382a86a2f645b4857aa04e459209d9 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:43.163831 28799 ts_tablet_manager.cc:1403] T c9382a86a2f645b4857aa04e459209d9 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.002s
I20260618 13:59:43.164063 30777 ts_tablet_manager.cc:1929] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:43.164170 30777 log.cc:1199] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/240c1f7ac1ee4fcb9e48dc3d4bbc4dd3
I20260618 13:59:43.164506 30777 ts_tablet_manager.cc:1950] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:43.164669 28799 raft_consensus.cc:359] T c9382a86a2f645b4857aa04e459209d9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.164862 28799 raft_consensus.cc:385] T c9382a86a2f645b4857aa04e459209d9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.164963 28799 raft_consensus.cc:740] T c9382a86a2f645b4857aa04e459209d9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.165220 28799 consensus_queue.cc:260] T c9382a86a2f645b4857aa04e459209d9 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.165515 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 (table default.Table_83d021ed148243ffb453cb5e2484b7b4 [id=b3cae943ecdf410ca1cf711c5ddfe14a]) successfully deleted
I20260618 13:59:43.166074 28799 ts_tablet_manager.cc:1434] T c9382a86a2f645b4857aa04e459209d9 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:43.166816 28799 tablet_bootstrap.cc:492] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:43.167375 30788 tablet.cc:1724] T 9ead386562eb4fd787c7aee776c2194f P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260618 13:59:43.168267 30152 raft_consensus.cc:493] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.168383 30152 raft_consensus.cc:515] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.168951 30152 leader_election.cc:290] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.170223 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df0bb1a20043049275fdc25084dc0d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:43.170825 28797 tablet_bootstrap.cc:492] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:43.170850 28799 tablet_bootstrap.cc:654] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.171026 28797 ts_tablet_manager.cc:1403] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.063s user 0.010s sys 0.002s
W20260618 13:59:43.171006 28346 leader_election.cc:343] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.170774 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df0bb1a20043049275fdc25084dc0d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:43.171960 28346 leader_election.cc:343] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.172070 28346 leader_election.cc:304] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.172422 30152 raft_consensus.cc:2749] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.172679 29746 rpcz_store.cc:276] 0618 13:59:40.193877 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.193932 (+ 55us) service_pool.cc:224] Handling call
0618 13:59:43.162801 (+2968869us) ts_tablet_manager.cc:1151] Deleting tablet e95c88e4b0a046a3ad69ee49af47c7c3
0618 13:59:43.162810 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.162862 (+ 52us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2967644}
W20260618 13:59:43.174510 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet e95c88e4b0a046a3ad69ee49af47c7c3 because the tablet was not found. No further retry: Not found: Tablet not found: e95c88e4b0a046a3ad69ee49af47c7c3
I20260618 13:59:43.174885 29746 tablet_replica.cc:333] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:43.174944 28797 raft_consensus.cc:359] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.175176 28797 raft_consensus.cc:385] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.175257 28797 raft_consensus.cc:740] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.175508 28797 consensus_queue.cc:260] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.175839 29026 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 135) took 1860 ms. Trace:
I20260618 13:59:43.176743 29026 rpcz_store.cc:276] 0618 13:59:41.315419 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:43.093549 (+1778130us) service_pool.cc:224] Handling call
0618 13:59:43.123540 (+ 29991us) spinlock_profiling.cc:243] Waited 29.7 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb46846c3 00007fcbab0e07c6 00007fcbab0e04f5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:43.124431 (+ 891us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:43.124986 (+ 555us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:43.175533 (+ 50547us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:43.175668 (+ 135us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:43.175796 (+ 128us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:43.126024 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:43.126152 (+ 128us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:43.126160 (+ 8us) write_op.cc:435] Acquired schema lock
0618 13:59:43.126164 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:43.126271 (+ 107us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:43.126283 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:43.126290 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:43.126386 (+ 96us) tablet.cc:701] Row locks acquired
0618 13:59:43.126390 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:43.126505 (+ 115us) write_op.cc:270] Start()
0618 13:59:43.126542 (+ 37us) write_op.cc:276] Timestamp: P: 1781791183126388 usec, L: 0
0618 13:59:43.126548 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:43.126729 (+ 181us) log.cc:844] Serialized 433 byte log entry
0618 13:59:43.172903 (+ 46174us) op_driver.cc:464] REPLICATION: finished
0618 13:59:43.173906 (+ 1003us) write_op.cc:301] APPLY: starting
0618 13:59:43.173914 (+ 8us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:43.173940 (+ 26us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:43.173958 (+ 18us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:43.174046 (+ 88us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:43.174070 (+ 24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:43.174076 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:43.174121 (+ 45us) log.cc:844] Serialized 29 byte log entry
0618 13:59:43.174162 (+ 41us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:43.174177 (+ 15us) write_op.cc:454] Released schema lock
0618 13:59:43.174182 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":68321664,"child_traces":[["op",{"apply.queue_time_us":974,"apply.run_cpu_time_us":328,"apply.run_wall_time_us":327,"num_ops":2,"prepare.queue_time_us":49,"prepare.run_cpu_time_us":796,"prepare.run_wall_time_us":817,"raft.queue_time_us":85,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":46338,"threads_started":1}]]}
I20260618 13:59:43.178066 29746 raft_consensus.cc:2243] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:43.178661 28730 heartbeater.cc:344] Connected to a master server at 127.27.100.61:45967
I20260618 13:59:43.178799 28730 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:43.182267 29566 ts_tablet_manager.cc:1929] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:43.182380 29566 log.cc:1199] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/3dd0eaa64271489d92aa380df877f257
I20260618 13:59:43.182881 30719 raft_consensus.cc:493] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.183043 30719 raft_consensus.cc:515] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.185065 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8da96fc621c47d4a444b5a39259f85d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:43.185482 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8da96fc621c47d4a444b5a39259f85d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:43.185688 28411 raft_consensus.cc:2468] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:43.186272 28618 leader_election.cc:343] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.186787 28617 leader_election.cc:304] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:43.187045 30584 raft_consensus.cc:2804] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:43.187141 30584 raft_consensus.cc:493] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:43.187196 30584 raft_consensus.cc:3060] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.188125 29746 pending_rounds.cc:70] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2: Trying to abort 1 pending ops.
I20260618 13:59:43.188198 29746 pending_rounds.cc:77] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298216683262812160 op_type: NO_OP noop_request { }
I20260618 13:59:43.188306 29746 raft_consensus.cc:2889] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:43.188406 29746 raft_consensus.cc:2272] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:43.189757 30584 raft_consensus.cc:515] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.190685 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8da96fc621c47d4a444b5a39259f85d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:43.191162 28618 leader_election.cc:343] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.190935 30584 leader_election.cc:290] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:43.191810 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8da96fc621c47d4a444b5a39259f85d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:43.191967 28411 raft_consensus.cc:3060] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.195122 29026 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_819e5138e3c7415a80631f3cbb6f8b81" } new_table_name: "default.Table_21426dbf385d47d2af5fc25e287ed0ed" modify_external_catalogs: true
I20260618 13:59:43.196798 28411 raft_consensus.cc:2468] T a8da96fc621c47d4a444b5a39259f85d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:43.192245 29746 ts_tablet_manager.cc:1916] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:43.194537 30774 tablet.cc:1724] T 9ead386562eb4fd787c7aee776c2194f P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260618 13:59:43.197731 28617 leader_election.cc:304] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:43.198199 30577 raft_consensus.cc:2804] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:43.198307 30577 raft_consensus.cc:697] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:43.198536 30577 consensus_queue.cc:237] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.200939 30584 raft_consensus.cc:493] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.201095 30584 raft_consensus.cc:515] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.201786 30777 tablet_replica.cc:333] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:43.202157 30777 raft_consensus.cc:2243] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:43.202958 30777 raft_consensus.cc:2272] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:43.203763 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bef52c0fc354d6cb5388700641af9f8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:43.204236 28618 leader_election.cc:343] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.205348 30584 leader_election.cc:290] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:43.205531 28548 raft_consensus.cc:3060] T fe1139c409e84070a2f496a85545e8d5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.205688 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bef52c0fc354d6cb5388700641af9f8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:43.205842 28411 raft_consensus.cc:2468] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:43.206228 28617 leader_election.cc:304] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:43.206607 30584 raft_consensus.cc:2804] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:43.206669 30584 raft_consensus.cc:493] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:43.206717 30584 raft_consensus.cc:3060] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.207317 29303 raft_consensus.cc:493] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.207424 29303 raft_consensus.cc:515] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.207897 29303 leader_election.cc:290] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:43.208562 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db62b3a1e37b447c9ea3fbd65c0cf1e7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:43.208876 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db62b3a1e37b447c9ea3fbd65c0cf1e7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:43.209036 28411 raft_consensus.cc:2468] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:43.209074 28540 raft_consensus.cc:3060] T f1663f9d92e54f1caf3c7137412e495c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.209442 28617 leader_election.cc:304] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:43.209652 29303 raft_consensus.cc:2804] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:43.209705 29303 raft_consensus.cc:493] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:43.209771 29303 raft_consensus.cc:3060] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.212169 29303 raft_consensus.cc:515] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:43.214890 28618 leader_election.cc:343] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.215497 29303 leader_election.cc:290] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:43.216043 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db62b3a1e37b447c9ea3fbd65c0cf1e7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:43.218719 28618 leader_election.cc:343] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.226635 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db62b3a1e37b447c9ea3fbd65c0cf1e7" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:43.226955 28411 raft_consensus.cc:3060] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.227131 30719 leader_election.cc:290] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:43.230195 28411 raft_consensus.cc:2468] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:43.232939 30584 raft_consensus.cc:515] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.239928 28545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bef52c0fc354d6cb5388700641af9f8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:43.239943 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bef52c0fc354d6cb5388700641af9f8" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
W20260618 13:59:43.240542 28618 leader_election.cc:343] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.240726 28403 raft_consensus.cc:3060] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.243822 28403 raft_consensus.cc:2468] T 8bef52c0fc354d6cb5388700641af9f8 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:43.244526 28617 leader_election.cc:304] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:43.244925 30719 raft_consensus.cc:2804] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:43.245105 30719 raft_consensus.cc:697] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:43.245469 30719 consensus_queue.cc:237] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.249104 28617 leader_election.cc:304] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:43.249408 30719 raft_consensus.cc:2804] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:43.249584 30584 leader_election.cc:290] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:43.249874 30719 raft_consensus.cc:697] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:43.250149 30719 consensus_queue.cc:237] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.254573 30777 ts_tablet_manager.cc:1916] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:43.259332 28797 ts_tablet_manager.cc:1434] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.085s user 0.001s sys 0.000s
I20260618 13:59:43.262548 28797 tablet_bootstrap.cc:492] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:43.263669 28798 tablet_bootstrap.cc:492] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:43.263916 28798 ts_tablet_manager.cc:1403] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.162s user 0.006s sys 0.002s
I20260618 13:59:43.264792 28798 raft_consensus.cc:359] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.265111 28798 raft_consensus.cc:385] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.265233 28798 raft_consensus.cc:740] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.265497 28798 consensus_queue.cc:260] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.266211 28798 ts_tablet_manager.cc:1434] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:43.266619 28798 tablet_bootstrap.cc:492] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:43.268803 28798 tablet_bootstrap.cc:654] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.271602 28797 tablet_bootstrap.cc:654] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.282884 29566 ts_tablet_manager.cc:1950] T 3dd0eaa64271489d92aa380df877f257 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:43.283955 28799 tablet_bootstrap.cc:492] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:43.284740 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 186) took 8305 ms. Trace:
I20260618 13:59:43.284968 28799 ts_tablet_manager.cc:1403] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.119s user 0.017s sys 0.020s
I20260618 13:59:43.285645 28799 raft_consensus.cc:359] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:43.285876 29566 rpcz_store.cc:276] 0618 13:59:34.979441 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.979487 (+ 46us) service_pool.cc:224] Handling call
0618 13:59:43.124955 (+8145468us) ts_tablet_manager.cc:1151] Deleting tablet 3dd0eaa64271489d92aa380df877f257
0618 13:59:43.124965 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.158174 (+ 33209us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.182215 (+ 24041us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.284723 (+102508us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8145303}
I20260618 13:59:43.286458 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 3dd0eaa64271489d92aa380df877f257 (table default.Table_fd6ef45aeec84b48b8fb7a54db4b4f81 [id=ca38f91f52db4c69a0fb3bc6b8f5cadc]) successfully deleted
I20260618 13:59:43.286559 28797 tablet_bootstrap.cc:492] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:43.286675 28797 ts_tablet_manager.cc:1403] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.000s
I20260618 13:59:43.286914 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 187) took 8307 ms. Trace:
I20260618 13:59:43.287027 29566 rpcz_store.cc:276] 0618 13:59:34.979848 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.979893 (+ 45us) service_pool.cc:224] Handling call
0618 13:59:43.286828 (+8306935us) ts_tablet_manager.cc:1151] Deleting tablet 316f470ea82f42fca48e4e96d49c9574
0618 13:59:43.286837 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.286899 (+ 62us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8306740}
I20260618 13:59:43.287276 28799 raft_consensus.cc:385] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.287348 28797 raft_consensus.cc:359] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.287520 28797 raft_consensus.cc:385] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.287575 28797 raft_consensus.cc:740] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
W20260618 13:59:43.287693 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 316f470ea82f42fca48e4e96d49c9574 because the tablet was not found. No further retry: Not found: Tablet not found: 316f470ea82f42fca48e4e96d49c9574
I20260618 13:59:43.287894 28797 consensus_queue.cc:260] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.288677 28797 ts_tablet_manager.cc:1434] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:43.288937 28797 tablet_bootstrap.cc:492] T ac9f580e35af4d4080d313b0283c0b7e P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:43.291661 28797 tablet_bootstrap.cc:654] T ac9f580e35af4d4080d313b0283c0b7e P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.298110 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 188) took 8317 ms. Trace:
I20260618 13:59:43.298287 28799 raft_consensus.cc:740] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.298609 28799 consensus_queue.cc:260] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:43.298846 29566 rpcz_store.cc:276] 0618 13:59:34.980255 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.980292 (+ 37us) service_pool.cc:224] Handling call
0618 13:59:43.298009 (+8317717us) ts_tablet_manager.cc:1151] Deleting tablet 2188ebf1727240cd886ba7662f8fcf43
0618 13:59:43.298020 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.298091 (+ 71us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8317540}
W20260618 13:59:43.299386 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 2188ebf1727240cd886ba7662f8fcf43 because the tablet was not found. No further retry: Not found: Tablet not found: 2188ebf1727240cd886ba7662f8fcf43
I20260618 13:59:43.300529 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 189) took 8319 ms. Trace:
I20260618 13:59:43.300760 29566 rpcz_store.cc:276] 0618 13:59:34.980620 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.980844 (+ 224us) service_pool.cc:224] Handling call
0618 13:59:43.300443 (+8319599us) ts_tablet_manager.cc:1151] Deleting tablet 3a0039a463ec4a9c82a21a3a69b565fc
0618 13:59:43.300453 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.300514 (+ 61us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8319391}
I20260618 13:59:43.299993 28799 ts_tablet_manager.cc:1434] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.015s user 0.001s sys 0.000s
W20260618 13:59:43.301342 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 3a0039a463ec4a9c82a21a3a69b565fc because the tablet was not found. No further retry: Not found: Tablet not found: 3a0039a463ec4a9c82a21a3a69b565fc
I20260618 13:59:43.302434 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 191) took 8320 ms. Trace:
I20260618 13:59:43.302549 28799 tablet_bootstrap.cc:492] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:43.302627 29566 rpcz_store.cc:276] 0618 13:59:34.981515 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.981558 (+ 43us) service_pool.cc:224] Handling call
0618 13:59:43.302179 (+8320621us) ts_tablet_manager.cc:1151] Deleting tablet 8fd9fa9675234db2abff694a517c0ea8
0618 13:59:43.302188 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.302240 (+ 52us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8320393}
W20260618 13:59:43.303701 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 8fd9fa9675234db2abff694a517c0ea8 because the tablet was not found. No further retry: Not found: Tablet not found: 8fd9fa9675234db2abff694a517c0ea8
I20260618 13:59:43.304052 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 192) took 8322 ms. Trace:
I20260618 13:59:43.304164 29566 rpcz_store.cc:276] 0618 13:59:34.981967 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.982014 (+ 47us) service_pool.cc:224] Handling call
0618 13:59:43.303977 (+8321963us) ts_tablet_manager.cc:1151] Deleting tablet 092a9aba9c6e48f5be9cf5d55ba93678
0618 13:59:43.303986 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.304038 (+ 52us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8321814}
W20260618 13:59:43.306071 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 092a9aba9c6e48f5be9cf5d55ba93678 because the tablet was not found. No further retry: Not found: Tablet not found: 092a9aba9c6e48f5be9cf5d55ba93678
I20260618 13:59:43.306509 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 195) took 8323 ms. Trace:
I20260618 13:59:43.306625 29566 rpcz_store.cc:276] 0618 13:59:34.983420 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.983462 (+ 42us) service_pool.cc:224] Handling call
0618 13:59:43.306388 (+8322926us) ts_tablet_manager.cc:1151] Deleting tablet 41107eb1d9164a9c98060003b4027412
0618 13:59:43.306397 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.306492 (+ 95us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8322733}
W20260618 13:59:43.307118 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 41107eb1d9164a9c98060003b4027412 because the tablet was not found. No further retry: Not found: Tablet not found: 41107eb1d9164a9c98060003b4027412
I20260618 13:59:43.307432 28799 tablet_bootstrap.cc:654] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.307469 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 196) took 8323 ms. Trace:
I20260618 13:59:43.307621 29566 rpcz_store.cc:276] 0618 13:59:34.983825 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.983879 (+ 54us) service_pool.cc:224] Handling call
0618 13:59:43.307378 (+8323499us) ts_tablet_manager.cc:1151] Deleting tablet aef7c1f111634c98af046e73fa9ffdeb
0618 13:59:43.307388 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.307455 (+ 67us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8323307}
W20260618 13:59:43.308104 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet aef7c1f111634c98af046e73fa9ffdeb because the tablet was not found. No further retry: Not found: Tablet not found: aef7c1f111634c98af046e73fa9ffdeb
I20260618 13:59:43.308447 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 197) took 8324 ms. Trace:
I20260618 13:59:43.308560 29566 rpcz_store.cc:276] 0618 13:59:34.984252 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.984301 (+ 49us) service_pool.cc:224] Handling call
0618 13:59:43.308350 (+8324049us) ts_tablet_manager.cc:1151] Deleting tablet 233040a16b0d4b61b977ca57c1afec2d
0618 13:59:43.308361 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.308430 (+ 69us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8323862}
W20260618 13:59:43.309037 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 233040a16b0d4b61b977ca57c1afec2d because the tablet was not found. No further retry: Not found: Tablet not found: 233040a16b0d4b61b977ca57c1afec2d
I20260618 13:59:43.309342 28798 tablet_bootstrap.cc:492] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:43.309361 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 198) took 8324 ms. Trace:
I20260618 13:59:43.309486 28798 ts_tablet_manager.cc:1403] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.043s user 0.001s sys 0.002s
I20260618 13:59:43.309522 29566 rpcz_store.cc:276] 0618 13:59:34.984639 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.984677 (+ 38us) service_pool.cc:224] Handling call
0618 13:59:43.309292 (+8324615us) ts_tablet_manager.cc:1151] Deleting tablet 8b88ed7cf79c4929a8704f66691cbed4
0618 13:59:43.309299 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.309347 (+ 48us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8324458}
W20260618 13:59:43.310068 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 8b88ed7cf79c4929a8704f66691cbed4 because the tablet was not found. No further retry: Not found: Tablet not found: 8b88ed7cf79c4929a8704f66691cbed4
I20260618 13:59:43.310242 28798 raft_consensus.cc:359] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.310406 28798 raft_consensus.cc:385] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.310487 28798 raft_consensus.cc:740] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.310689 28798 consensus_queue.cc:260] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.311180 28798 ts_tablet_manager.cc:1434] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 13:59:43.311437 28798 tablet_bootstrap.cc:492] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:43.310524 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 199) took 8325 ms. Trace:
I20260618 13:59:43.311975 29566 rpcz_store.cc:276] 0618 13:59:34.985003 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.985052 (+ 49us) service_pool.cc:224] Handling call
0618 13:59:43.310317 (+8325265us) ts_tablet_manager.cc:1151] Deleting tablet b6dd422c05324c989e29b20ae2afabad
0618 13:59:43.310325 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.310374 (+ 49us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8325097}
W20260618 13:59:43.312914 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet b6dd422c05324c989e29b20ae2afabad because the tablet was not found. No further retry: Not found: Tablet not found: b6dd422c05324c989e29b20ae2afabad
I20260618 13:59:43.313256 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 200) took 8327 ms. Trace:
I20260618 13:59:43.313366 29566 rpcz_store.cc:276] 0618 13:59:34.985372 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.985412 (+ 40us) service_pool.cc:224] Handling call
0618 13:59:43.313182 (+8327770us) ts_tablet_manager.cc:1151] Deleting tablet d9c00003300246ada2a356e3e1cf684b
0618 13:59:43.313191 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.313241 (+ 50us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8327619}
I20260618 13:59:43.313583 28798 tablet_bootstrap.cc:654] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:43.313907 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d9c00003300246ada2a356e3e1cf684b because the tablet was not found. No further retry: Not found: Tablet not found: d9c00003300246ada2a356e3e1cf684b
I20260618 13:59:43.314213 29566 tablet_replica.cc:333] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:43.314484 29566 raft_consensus.cc:2243] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:43.318591 29566 raft_consensus.cc:2272] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:43.319332 29566 ts_tablet_manager.cc:1916] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:43.325779 29746 ts_tablet_manager.cc:1929] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:43.325917 29746 log.cc:1199] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/e5f7eaa7d218440ca2c35340c5c87814
I20260618 13:59:43.328405 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:43.328835 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:43.329155 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_93fe53cac2f54d73b1e66cf1661ca928 [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 3 got 2
I20260618 13:59:43.329442 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:43.336526 29566 ts_tablet_manager.cc:1929] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:43.336651 29566 log.cc:1199] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/c4cfca9e76e641a7bc0b58b9ee240f36
I20260618 13:59:43.337050 29566 ts_tablet_manager.cc:1950] T c4cfca9e76e641a7bc0b58b9ee240f36 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:43.337947 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 201) took 8352 ms. Trace:
I20260618 13:59:43.338039 29566 rpcz_store.cc:276] 0618 13:59:34.985907 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.985949 (+ 42us) service_pool.cc:224] Handling call
0618 13:59:43.314164 (+8328215us) ts_tablet_manager.cc:1151] Deleting tablet c4cfca9e76e641a7bc0b58b9ee240f36
0618 13:59:43.314174 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.333249 (+ 19075us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.336468 (+ 3219us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.337929 (+ 1461us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8328018}
I20260618 13:59:43.338533 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet c4cfca9e76e641a7bc0b58b9ee240f36 (table default.Table_0e9f574416d84a0f826fa9a1fcc56210 [id=098fa24bb71e4fe1aecc33aea1275ef0]) successfully deleted
I20260618 13:59:43.341825 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 202) took 8353 ms. Trace:
I20260618 13:59:43.341955 29566 rpcz_store.cc:276] 0618 13:59:34.988258 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:34.988301 (+ 43us) service_pool.cc:224] Handling call
0618 13:59:43.341717 (+8353416us) ts_tablet_manager.cc:1151] Deleting tablet 897bf6d6bbee4ad89c296325d94a3e53
0618 13:59:43.341728 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.341809 (+ 81us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8353235}
W20260618 13:59:43.342465 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 897bf6d6bbee4ad89c296325d94a3e53 because the tablet was not found. No further retry: Not found: Tablet not found: 897bf6d6bbee4ad89c296325d94a3e53
I20260618 13:59:43.342803 29566 tablet_replica.cc:333] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:43.343024 29566 raft_consensus.cc:2243] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:43.344103 28799 tablet_bootstrap.cc:492] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:43.344283 28799 ts_tablet_manager.cc:1403] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.043s user 0.002s sys 0.011s
I20260618 13:59:43.344642 29566 raft_consensus.cc:2272] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:43.345188 29566 ts_tablet_manager.cc:1916] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:43.345297 28799 raft_consensus.cc:359] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.345557 28799 raft_consensus.cc:385] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.345638 28799 raft_consensus.cc:740] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.345906 28799 consensus_queue.cc:260] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.347280 28799 ts_tablet_manager.cc:1434] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260618 13:59:43.329031 28798 tablet_bootstrap.cc:492] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:43.351099 28798 ts_tablet_manager.cc:1403] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.040s user 0.000s sys 0.006s
I20260618 13:59:43.351176 28799 tablet_bootstrap.cc:492] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:43.351866 28798 raft_consensus.cc:359] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.352041 28798 raft_consensus.cc:385] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.352097 28798 raft_consensus.cc:740] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.352293 28798 consensus_queue.cc:260] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.352790 28798 ts_tablet_manager.cc:1434] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:43.353089 28798 tablet_bootstrap.cc:492] T eebebb79db644535b54a48e4dee89d59 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:43.355268 28798 tablet_bootstrap.cc:654] T eebebb79db644535b54a48e4dee89d59 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.326404 29746 ts_tablet_manager.cc:1950] T e5f7eaa7d218440ca2c35340c5c87814 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:43.357903 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 246) took 2284 ms. Trace:
I20260618 13:59:43.363029 29746 rpcz_store.cc:276] 0618 13:59:41.073146 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.073197 (+ 51us) service_pool.cc:224] Handling call
0618 13:59:43.174820 (+2101623us) ts_tablet_manager.cc:1151] Deleting tablet e5f7eaa7d218440ca2c35340c5c87814
0618 13:59:43.174834 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.294582 (+119748us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.325704 (+ 31122us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.357885 (+ 32181us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2101462}
I20260618 13:59:43.363483 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet e5f7eaa7d218440ca2c35340c5c87814 (table default.Table_9a78338c3e414b0abea1281082202f57 [id=5697846ccd9c4c659755a0f9b8a66b72]) successfully deleted
I20260618 13:59:43.364212 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 248) took 2290 ms. Trace:
I20260618 13:59:43.364327 28799 tablet_bootstrap.cc:654] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.364630 29746 rpcz_store.cc:276] 0618 13:59:41.073316 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.073394 (+ 78us) service_pool.cc:224] Handling call
0618 13:59:43.364115 (+2290721us) ts_tablet_manager.cc:1151] Deleting tablet ca2ee746c5024cfc9b31c470f9616e7d
0618 13:59:43.364124 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.364196 (+ 72us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2290602}
W20260618 13:59:43.365309 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet ca2ee746c5024cfc9b31c470f9616e7d because the tablet was not found. No further retry: Not found: Tablet not found: ca2ee746c5024cfc9b31c470f9616e7d
I20260618 13:59:43.365731 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 250) took 2292 ms. Trace:
I20260618 13:59:43.366065 29746 rpcz_store.cc:276] 0618 13:59:41.073488 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.073538 (+ 50us) service_pool.cc:224] Handling call
0618 13:59:43.365651 (+2292113us) ts_tablet_manager.cc:1151] Deleting tablet 3dd0eaa64271489d92aa380df877f257
0618 13:59:43.365659 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.365714 (+ 55us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2292009}
I20260618 13:59:43.366554 30622 raft_consensus.cc:493] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260618 13:59:43.366717 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 3dd0eaa64271489d92aa380df877f257 because the tablet was not found. No further retry: Not found: Tablet not found: 3dd0eaa64271489d92aa380df877f257
I20260618 13:59:43.366700 30622 raft_consensus.cc:515] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:43.367030 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 251) took 2293 ms. Trace:
I20260618 13:59:43.367120 29746 rpcz_store.cc:276] 0618 13:59:41.073577 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.073667 (+ 90us) service_pool.cc:224] Handling call
0618 13:59:43.366959 (+2293292us) ts_tablet_manager.cc:1151] Deleting tablet ab521028ac1a4d0da013112d76bb169f
0618 13:59:43.366967 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.367015 (+ 48us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2293174}
I20260618 13:59:43.367300 30622 leader_election.cc:290] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:43.367494 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet ab521028ac1a4d0da013112d76bb169f because the tablet was not found. No further retry: Not found: Tablet not found: ab521028ac1a4d0da013112d76bb169f
I20260618 13:59:43.368168 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 253) took 2294 ms. Trace:
I20260618 13:59:43.368350 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76f83f23f22345bb9bc3f239642ea673" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:43.368556 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76f83f23f22345bb9bc3f239642ea673" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:43.368518 29746 rpcz_store.cc:276] 0618 13:59:41.073746 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.073806 (+ 60us) service_pool.cc:224] Handling call
0618 13:59:43.368097 (+2294291us) ts_tablet_manager.cc:1151] Deleting tablet ac58e939730c4af681c2f08bcfbb6602
0618 13:59:43.368106 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.368153 (+ 47us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2294188}
W20260618 13:59:43.368906 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet ac58e939730c4af681c2f08bcfbb6602 because the tablet was not found. No further retry: Not found: Tablet not found: ac58e939730c4af681c2f08bcfbb6602
W20260618 13:59:43.369036 28346 leader_election.cc:343] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:43.369266 28346 leader_election.cc:343] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.369413 28346 leader_election.cc:304] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.369743 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 244) took 2296 ms. Trace:
I20260618 13:59:43.370039 29746 rpcz_store.cc:276] 0618 13:59:41.072941 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.073885 (+ 944us) service_pool.cc:224] Handling call
0618 13:59:43.369513 (+2295628us) ts_tablet_manager.cc:1151] Deleting tablet f2b21dcf74fa4264947ca43cc0026e76
0618 13:59:43.369635 (+ 122us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.369727 (+ 92us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2295427}
I20260618 13:59:43.370544 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c9ceb3af08fa4618b81cf6f89d7bd0df (table default.Table_21242eba3cb64c7c91082e296439cfe2 [id=10a4f5ef579746b68d8dd37d83f14c3e]). Expected version 1 got 0
W20260618 13:59:43.370709 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet f2b21dcf74fa4264947ca43cc0026e76 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2b21dcf74fa4264947ca43cc0026e76 already in progress: creating tablet
I20260618 13:59:43.371006 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:43.371093 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 254) took 2297 ms. Trace:
I20260618 13:59:43.371160 30622 raft_consensus.cc:2749] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.371166 29746 rpcz_store.cc:276] 0618 13:59:41.073836 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.074005 (+ 169us) service_pool.cc:224] Handling call
0618 13:59:43.371022 (+2297017us) ts_tablet_manager.cc:1151] Deleting tablet 015494374645469785f1f978eaf0e9a5
0618 13:59:43.371030 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.371078 (+ 48us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2296911}
W20260618 13:59:43.371554 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 015494374645469785f1f978eaf0e9a5 because the tablet was not found. No further retry: Not found: Tablet not found: 015494374645469785f1f978eaf0e9a5
I20260618 13:59:43.372231 29746 tablet_replica.cc:333] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:43.372306 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet ba53ad1988064049a61fcca3a85f8779 (table default.Table_b7f097c1abe1410a800ce5aa25771740 [id=6c30b030a60b47e2be48dfe6e9cfdfdb]). Expected version 1 got 0
I20260618 13:59:43.372589 29746 raft_consensus.cc:2243] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:43.372735 29746 raft_consensus.cc:2272] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:43.373086 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet e196dd3c7fe14524a50152643ddbc2ad (table default.Table_2dfebd62a146446f999f1a7a1bd1af79 [id=02ffed3d852845d6845fb5176f036fce]). Expected version 1 got 0
I20260618 13:59:43.373235 29746 ts_tablet_manager.cc:1916] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:43.373721 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:43.375231 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:43.375502 30777 ts_tablet_manager.cc:1929] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:43.375689 30777 log.cc:1199] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/fea86899d2c047d997cebe80551e83c9
I20260618 13:59:43.376112 30777 ts_tablet_manager.cc:1950] T fea86899d2c047d997cebe80551e83c9 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:43.377252 29566 ts_tablet_manager.cc:1929] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:43.377385 29566 log.cc:1199] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/f3ba9186b3ff47d5a2b158fef619686e
I20260618 13:59:43.377591 30777 tablet_replica.cc:333] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:43.377821 30777 raft_consensus.cc:2243] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:43.377848 29566 ts_tablet_manager.cc:1950] T f3ba9186b3ff47d5a2b158fef619686e P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:43.378029 30777 raft_consensus.cc:2272] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:43.378510 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 203) took 8362 ms. Trace:
I20260618 13:59:43.378597 29566 rpcz_store.cc:276] 0618 13:59:35.015706 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.015760 (+ 54us) service_pool.cc:224] Handling call
0618 13:59:43.342756 (+8326996us) ts_tablet_manager.cc:1151] Deleting tablet f3ba9186b3ff47d5a2b158fef619686e
0618 13:59:43.342764 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.374284 (+ 31520us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.377183 (+ 2899us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.378493 (+ 1310us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8326736}
I20260618 13:59:43.378731 30777 ts_tablet_manager.cc:1916] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:43.380499 28799 tablet_bootstrap.cc:492] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:43.380622 28799 ts_tablet_manager.cc:1403] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.030s user 0.000s sys 0.009s
I20260618 13:59:43.381301 28799 raft_consensus.cc:359] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.381462 28799 raft_consensus.cc:385] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.381511 28799 raft_consensus.cc:740] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.381704 28799 consensus_queue.cc:260] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.382294 28799 ts_tablet_manager.cc:1434] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:43.382563 28799 tablet_bootstrap.cc:492] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:43.384238 28799 tablet_bootstrap.cc:654] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.385962 28798 tablet_bootstrap.cc:492] T eebebb79db644535b54a48e4dee89d59 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:43.386085 28798 ts_tablet_manager.cc:1403] T eebebb79db644535b54a48e4dee89d59 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.007s
I20260618 13:59:43.386745 28798 raft_consensus.cc:359] T eebebb79db644535b54a48e4dee89d59 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.386942 28798 raft_consensus.cc:385] T eebebb79db644535b54a48e4dee89d59 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.387007 28798 raft_consensus.cc:740] T eebebb79db644535b54a48e4dee89d59 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.387261 28798 consensus_queue.cc:260] T eebebb79db644535b54a48e4dee89d59 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.387527 28797 tablet_bootstrap.cc:492] T ac9f580e35af4d4080d313b0283c0b7e P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:43.387658 28797 ts_tablet_manager.cc:1403] T ac9f580e35af4d4080d313b0283c0b7e P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.099s user 0.021s sys 0.007s
I20260618 13:59:43.388331 28798 ts_tablet_manager.cc:1434] T eebebb79db644535b54a48e4dee89d59 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:43.388701 28797 raft_consensus.cc:359] T ac9f580e35af4d4080d313b0283c0b7e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.388881 28797 raft_consensus.cc:385] T ac9f580e35af4d4080d313b0283c0b7e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.388948 28797 raft_consensus.cc:740] T ac9f580e35af4d4080d313b0283c0b7e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.377161 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet fea86899d2c047d997cebe80551e83c9 (table default.Table_539ac216b6f4478eb994218d7d7980a4 [id=baa3214376034e9cae926def4c46b91e]) successfully deleted
I20260618 13:59:43.378961 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet f3ba9186b3ff47d5a2b158fef619686e (table default.Table_3ef2ff183a6044e2a9e069a4eac133b8 [id=60e91b2138bf41a5a8dee65c1874678a]) successfully deleted
I20260618 13:59:43.392714 28798 tablet_bootstrap.cc:492] T 8f7b66652f464893ad812d27e883a9a4 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:43.394306 28798 tablet_bootstrap.cc:654] T 8f7b66652f464893ad812d27e883a9a4 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.396262 30152 raft_consensus.cc:493] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.396407 30152 raft_consensus.cc:515] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.389181 28797 consensus_queue.cc:260] T ac9f580e35af4d4080d313b0283c0b7e P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.397032 30152 leader_election.cc:290] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.397143 29746 ts_tablet_manager.cc:1929] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:43.397240 29746 log.cc:1199] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/97a9f949999446f09359629fe483bf67
I20260618 13:59:43.397524 29746 ts_tablet_manager.cc:1950] T 97a9f949999446f09359629fe483bf67 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:43.397888 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc88bf849a94cb9bd1c9c14aec53eb1" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:43.398013 28797 ts_tablet_manager.cc:1434] T ac9f580e35af4d4080d313b0283c0b7e P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260618 13:59:43.398128 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc88bf849a94cb9bd1c9c14aec53eb1" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:43.398296 28797 tablet_bootstrap.cc:492] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:43.398864 28346 leader_election.cc:343] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:43.399080 28346 leader_election.cc:343] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.399046 30719 consensus_queue.cc:1048] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:43.399170 28346 leader_election.cc:304] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.399641 30152 raft_consensus.cc:2749] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.400861 30577 consensus_queue.cc:1048] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:43.401028 28797 tablet_bootstrap.cc:654] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.404685 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 245) took 2331 ms. Trace:
I20260618 13:59:43.404783 29746 rpcz_store.cc:276] 0618 13:59:41.073053 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.074118 (+ 1065us) service_pool.cc:224] Handling call
0618 13:59:43.372189 (+2298071us) ts_tablet_manager.cc:1151] Deleting tablet 97a9f949999446f09359629fe483bf67
0618 13:59:43.372197 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.376155 (+ 3958us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.397088 (+ 20933us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.404663 (+ 7575us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2297930}
I20260618 13:59:43.405196 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 247) took 2331 ms. Trace:
I20260618 13:59:43.405277 29746 rpcz_store.cc:276] 0618 13:59:41.073235 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.074291 (+ 1056us) service_pool.cc:224] Handling call
0618 13:59:43.405099 (+2330808us) ts_tablet_manager.cc:1151] Deleting tablet 283523e5b22843ba935eabe6283d3a99
0618 13:59:43.405109 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.405182 (+ 73us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2330665}
I20260618 13:59:43.405372 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 97a9f949999446f09359629fe483bf67 (table default.Table_9424eec16a284672a0fe7ab3d0852b40 [id=070ea48af78346aa8ff43f2bb369730c]) successfully deleted
I20260618 13:59:43.405582 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 252) took 2331 ms. Trace:
I20260618 13:59:43.405648 29746 rpcz_store.cc:276] 0618 13:59:41.073658 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.073933 (+ 275us) service_pool.cc:224] Handling call
0618 13:59:43.405508 (+2331575us) ts_tablet_manager.cc:1151] Deleting tablet 7572d17724d249739be1a1cb9468f909
0618 13:59:43.405517 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.405570 (+ 53us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2330967}
I20260618 13:59:43.405908 29746 tablet_replica.cc:333] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
W20260618 13:59:43.405853 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 283523e5b22843ba935eabe6283d3a99 because the tablet was not found. No further retry: Not found: Tablet not found: 283523e5b22843ba935eabe6283d3a99
I20260618 13:59:43.406104 29746 raft_consensus.cc:2243] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:43.406244 29746 raft_consensus.cc:2272] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:43.406463 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 7572d17724d249739be1a1cb9468f909 because the tablet was not found. No further retry: Not found: Tablet not found: 7572d17724d249739be1a1cb9468f909
I20260618 13:59:43.406785 29746 ts_tablet_manager.cc:1916] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:43.407275 30719 raft_consensus.cc:493] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.407433 30719 raft_consensus.cc:515] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.408087 30719 leader_election.cc:290] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:43.408654 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a71e334ed1400d97c965b49b677a81" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:43.408891 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a71e334ed1400d97c965b49b677a81" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:43.409037 28403 raft_consensus.cc:2468] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:43.409401 28618 leader_election.cc:343] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.409729 28617 leader_election.cc:304] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:43.410055 30152 raft_consensus.cc:493] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.410173 30152 raft_consensus.cc:515] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.410666 30152 leader_election.cc:290] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.410944 30719 raft_consensus.cc:2804] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:43.411012 30719 raft_consensus.cc:493] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:43.411065 30719 raft_consensus.cc:3060] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.411298 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6ba4976bb6e4c2da3a563ccad4eb2b0" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:43.411731 28346 leader_election.cc:343] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.413607 30719 raft_consensus.cc:515] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.414499 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6ba4976bb6e4c2da3a563ccad4eb2b0" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:43.415035 28346 leader_election.cc:343] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.415138 28346 leader_election.cc:304] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.415506 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a71e334ed1400d97c965b49b677a81" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:43.416404 28618 leader_election.cc:343] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.417800 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a71e334ed1400d97c965b49b677a81" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:43.418004 28403 raft_consensus.cc:3060] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.418373 30152 raft_consensus.cc:2749] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.418768 30719 leader_election.cc:290] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:43.418736 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_01e1e5741bb54f78996857f6b79fc5d7 [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 2 got 0
I20260618 13:59:43.419050 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:43.419376 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:43.420779 30152 raft_consensus.cc:493] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.420940 30152 raft_consensus.cc:515] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.421717 30152 leader_election.cc:290] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.422993 28799 tablet_bootstrap.cc:492] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:43.423136 28799 ts_tablet_manager.cc:1403] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.041s user 0.003s sys 0.000s
I20260618 13:59:43.423131 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cb78c3ca3b74b399d2b11320473e463" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:43.423673 28346 leader_election.cc:343] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.423947 28799 raft_consensus.cc:359] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.424120 28799 raft_consensus.cc:385] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.424186 28799 raft_consensus.cc:740] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.424396 28799 consensus_queue.cc:260] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.425009 28799 ts_tablet_manager.cc:1434] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:43.425283 28799 tablet_bootstrap.cc:492] T 25463905e343470d941f89a53efbc2d1 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:43.425426 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:43.425603 28403 raft_consensus.cc:2468] T d8a71e334ed1400d97c965b49b677a81 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:43.425856 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cb78c3ca3b74b399d2b11320473e463" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:43.426354 28346 leader_election.cc:343] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.426491 28346 leader_election.cc:304] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.426553 28617 leader_election.cc:304] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:43.426803 30152 raft_consensus.cc:2749] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.427241 30577 raft_consensus.cc:2804] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:43.427358 30577 raft_consensus.cc:697] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:43.427665 30577 consensus_queue.cc:237] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.431402 30719 raft_consensus.cc:493] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.431578 30719 raft_consensus.cc:515] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.431905 28799 tablet_bootstrap.cc:654] T 25463905e343470d941f89a53efbc2d1 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.432310 30719 leader_election.cc:290] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:43.432595 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c25fa765ec794c34b0aa43761961a8c9" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:43.432865 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c25fa765ec794c34b0aa43761961a8c9" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:43.433434 28403 raft_consensus.cc:2468] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:43.433851 28618 leader_election.cc:343] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.434195 28617 leader_election.cc:304] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:43.434634 30719 raft_consensus.cc:2804] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:43.434742 30719 raft_consensus.cc:493] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:43.434852 30719 raft_consensus.cc:3060] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.436144 30152 raft_consensus.cc:493] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.436299 30152 raft_consensus.cc:515] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.437345 30152 leader_election.cc:290] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.437711 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a662bb37fb7d4df49e61e524808b67a8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:43.437847 28680 raft_consensus.cc:2468] T a662bb37fb7d4df49e61e524808b67a8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:43.437865 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a662bb37fb7d4df49e61e524808b67a8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:43.438864 28346 leader_election.cc:304] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
W20260618 13:59:43.439196 28346 leader_election.cc:343] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.439687 30152 raft_consensus.cc:2804] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:43.439759 30152 raft_consensus.cc:493] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:43.439814 30152 raft_consensus.cc:3060] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.441974 29746 ts_tablet_manager.cc:1929] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:43.442093 29746 log.cc:1199] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/5e25027f423144b088dad955c7282921
I20260618 13:59:43.442209 30719 raft_consensus.cc:515] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.442396 29746 ts_tablet_manager.cc:1950] T 5e25027f423144b088dad955c7282921 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:43.442857 30152 raft_consensus.cc:515] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.443238 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 249) took 2369 ms. Trace:
I20260618 13:59:43.443353 29746 rpcz_store.cc:276] 0618 13:59:41.073411 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.073972 (+ 561us) service_pool.cc:224] Handling call
0618 13:59:43.405867 (+2331895us) ts_tablet_manager.cc:1151] Deleting tablet 5e25027f423144b088dad955c7282921
0618 13:59:43.405876 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.409990 (+ 4114us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.441910 (+ 31920us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.443218 (+ 1308us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2331256}
I20260618 13:59:43.443509 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c25fa765ec794c34b0aa43761961a8c9" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:43.443722 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 255) took 2353 ms. Trace:
I20260618 13:59:43.443805 29746 rpcz_store.cc:276] 0618 13:59:41.090268 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.093242 (+ 2974us) service_pool.cc:224] Handling call
0618 13:59:43.443650 (+2350408us) ts_tablet_manager.cc:1151] Deleting tablet 04dac931e62a448d99633878eb70b92f
0618 13:59:43.443659 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.443708 (+ 49us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2350229}
I20260618 13:59:43.444139 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 256) took 2350 ms. Trace:
I20260618 13:59:43.444212 29746 rpcz_store.cc:276] 0618 13:59:41.093534 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.093584 (+ 50us) service_pool.cc:224] Handling call
0618 13:59:43.444057 (+2350473us) ts_tablet_manager.cc:1151] Deleting tablet 9c17e76ae9b44207998a43e3576908c4
0618 13:59:43.444067 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.444122 (+ 55us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2350326}
I20260618 13:59:43.444263 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 5e25027f423144b088dad955c7282921 (table default.Table_a7ff69763fdb4ff3b26b180bf05907d9 [id=e39e2c5718794a748361d7b453c65404]) successfully deleted
W20260618 13:59:43.444325 28618 leader_election.cc:343] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.444469 29746 tablet_replica.cc:333] T 55ae29c3b7ed44cfa659130745037aa6 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:43.444418 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a662bb37fb7d4df49e61e524808b67a8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:43.444660 29746 raft_consensus.cc:2243] T 55ae29c3b7ed44cfa659130745037aa6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:43.444757 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c25fa765ec794c34b0aa43761961a8c9" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:43.444820 29746 raft_consensus.cc:2272] T 55ae29c3b7ed44cfa659130745037aa6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:43.444823 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 04dac931e62a448d99633878eb70b92f because the tablet was not found. No further retry: Not found: Tablet not found: 04dac931e62a448d99633878eb70b92f
W20260618 13:59:43.445042 28346 leader_election.cc:343] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.444945 28403 raft_consensus.cc:3060] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.445943 30152 leader_election.cc:290] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.446044 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a662bb37fb7d4df49e61e524808b67a8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:43.446234 28680 raft_consensus.cc:3060] T a662bb37fb7d4df49e61e524808b67a8 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.453306 28403 raft_consensus.cc:2468] T c25fa765ec794c34b0aa43761961a8c9 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:43.453732 28617 leader_election.cc:304] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:43.454003 30577 raft_consensus.cc:2804] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:43.454118 30577 raft_consensus.cc:697] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:43.454407 30577 consensus_queue.cc:237] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.463722 30777 ts_tablet_manager.cc:1929] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:43.425997 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:43.470559 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 205) took 8068 ms. Trace:
I20260618 13:59:43.470660 29566 rpcz_store.cc:276] 0618 13:59:35.402276 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.402338 (+ 62us) service_pool.cc:224] Handling call
0618 13:59:43.470409 (+8068071us) ts_tablet_manager.cc:1151] Deleting tablet d6871232860b41d489e34d95eb897241
0618 13:59:43.470459 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.470536 (+ 77us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8067828}
W20260618 13:59:43.471349 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d6871232860b41d489e34d95eb897241 because the tablet was not found. No further retry: Not found: Tablet not found: d6871232860b41d489e34d95eb897241
I20260618 13:59:43.471740 29566 tablet_replica.cc:333] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:43.471707 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:43.472008 29566 raft_consensus.cc:2243] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:43.472014 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 8af2bec820b842b4bf6b57482007bd6a (table default.Table_84be6926f4b64876b343577b85db3243 [id=af1859c644b44c1d831314d27a494f94]). Expected version 1 got 0
I20260618 13:59:43.472215 29566 raft_consensus.cc:2272] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:43.472214 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:43.472388 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:43.472790 29566 ts_tablet_manager.cc:1916] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:43.473227 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:43.473600 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:43.474661 30719 leader_election.cc:290] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:43.478005 30552 raft_consensus.cc:493] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.478148 30552 raft_consensus.cc:515] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.478770 30552 leader_election.cc:290] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:43.479794 28483 leader_election.cc:343] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: f45556a59c424ca2bc30956566374c89
I20260618 13:59:43.485263 30777 log.cc:1199] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/2128dbec33db453ba51070767a15b21e
I20260618 13:59:43.485719 30777 ts_tablet_manager.cc:1950] T 2128dbec33db453ba51070767a15b21e P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:43.487115 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 2128dbec33db453ba51070767a15b21e (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e]) successfully deleted
I20260618 13:59:43.488242 28798 tablet_bootstrap.cc:492] T 8f7b66652f464893ad812d27e883a9a4 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:43.488379 28798 ts_tablet_manager.cc:1403] T 8f7b66652f464893ad812d27e883a9a4 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.096s user 0.025s sys 0.017s
I20260618 13:59:43.488565 30152 raft_consensus.cc:493] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.488693 30152 raft_consensus.cc:515] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.489256 28798 raft_consensus.cc:359] T 8f7b66652f464893ad812d27e883a9a4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.489409 28798 raft_consensus.cc:385] T 8f7b66652f464893ad812d27e883a9a4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.489465 28798 raft_consensus.cc:740] T 8f7b66652f464893ad812d27e883a9a4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.489634 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e493f647a344cd581515a77230d765f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:43.489674 28798 consensus_queue.cc:260] T 8f7b66652f464893ad812d27e883a9a4 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.490069 28799 tablet_bootstrap.cc:492] T 25463905e343470d941f89a53efbc2d1 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:43.490201 28799 ts_tablet_manager.cc:1403] T 25463905e343470d941f89a53efbc2d1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.065s user 0.010s sys 0.009s
W20260618 13:59:43.490746 28346 leader_election.cc:343] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.491187 30152 leader_election.cc:290] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.491757 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e493f647a344cd581515a77230d765f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:43.491359 28799 raft_consensus.cc:359] T 25463905e343470d941f89a53efbc2d1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.492039 30262 raft_consensus.cc:493] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.492501 28799 raft_consensus.cc:385] T 25463905e343470d941f89a53efbc2d1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.492559 28799 raft_consensus.cc:740] T 25463905e343470d941f89a53efbc2d1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.492565 30262 raft_consensus.cc:515] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.492789 28799 consensus_queue.cc:260] T 25463905e343470d941f89a53efbc2d1 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.493095 30262 leader_election.cc:290] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.493278 28798 ts_tablet_manager.cc:1434] T 8f7b66652f464893ad812d27e883a9a4 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260618 13:59:43.493330 28799 ts_tablet_manager.cc:1434] T 25463905e343470d941f89a53efbc2d1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260618 13:59:43.493530 28798 tablet_bootstrap.cc:492] T f2b21dcf74fa4264947ca43cc0026e76 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:43.493597 28799 tablet_bootstrap.cc:492] T 66ada05b01b2427da182998a557529ca P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
W20260618 13:59:43.492369 28346 leader_election.cc:343] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.494674 28346 leader_election.cc:304] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.495494 28798 tablet_bootstrap.cc:654] T f2b21dcf74fa4264947ca43cc0026e76 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.495663 30622 raft_consensus.cc:2749] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.496085 28799 tablet_bootstrap.cc:654] T 66ada05b01b2427da182998a557529ca P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.496131 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00a2abfda3ef40eca1f30ff7a1ec050e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:43.496886 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00a2abfda3ef40eca1f30ff7a1ec050e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:43.497542 28346 leader_election.cc:343] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:43.497752 28346 leader_election.cc:343] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.497838 28346 leader_election.cc:304] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.498179 30622 raft_consensus.cc:2749] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.503176 29746 ts_tablet_manager.cc:1916] T 55ae29c3b7ed44cfa659130745037aa6 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:43.503558 30719 raft_consensus.cc:493] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.503700 30719 raft_consensus.cc:515] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.506109 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f45556a59c424ca2bc30956566374c89" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:43.510627 28484 leader_election.cc:343] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: Tablet not RUNNING: STOPPED
I20260618 13:59:43.510776 28484 leader_election.cc:304] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.511029 30552 raft_consensus.cc:2749] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.511210 30552 raft_consensus.cc:493] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.511300 30552 raft_consensus.cc:515] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.511798 30552 leader_election.cc:290] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:43.512585 28618 consensus_peers.cc:597] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:43.512567 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b2f34bc17b4a27a91d6cafa9a96c48" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:43.513176 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b2f34bc17b4a27a91d6cafa9a96c48" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:43.513722 28484 leader_election.cc:304] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.514173 30552 raft_consensus.cc:2749] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.518234 28680 raft_consensus.cc:2468] T a662bb37fb7d4df49e61e524808b67a8 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:43.520085 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3d8616846c649b6a969b971db0d03db" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:43.520812 29746 ts_tablet_manager.cc:1929] T 55ae29c3b7ed44cfa659130745037aa6 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:43.521057 29746 log.cc:1199] T 55ae29c3b7ed44cfa659130745037aa6 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/55ae29c3b7ed44cfa659130745037aa6
I20260618 13:59:43.521459 29746 ts_tablet_manager.cc:1950] T 55ae29c3b7ed44cfa659130745037aa6 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:43.522305 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 257) took 2428 ms. Trace:
I20260618 13:59:43.522393 29746 rpcz_store.cc:276] 0618 13:59:41.093622 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.093757 (+ 135us) service_pool.cc:224] Handling call
0618 13:59:43.444424 (+2350667us) ts_tablet_manager.cc:1151] Deleting tablet 55ae29c3b7ed44cfa659130745037aa6
0618 13:59:43.444434 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.509347 (+ 64913us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.520743 (+ 11396us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.522288 (+ 1545us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2350537}
I20260618 13:59:43.522535 30552 raft_consensus.cc:493] T fc295a8e81b64859915a6335f8fb8d8e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.522639 30552 raft_consensus.cc:515] T fc295a8e81b64859915a6335f8fb8d8e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.522883 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 55ae29c3b7ed44cfa659130745037aa6 (table default.Table_5d485acde41448d984aa79fa58b1866b [id=726324d365b94ff6bac24030222ca34f]) successfully deleted
I20260618 13:59:43.523206 30552 leader_election.cc:290] T fc295a8e81b64859915a6335f8fb8d8e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:43.518662 30719 leader_election.cc:290] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:43.519645 28617 leader_election.cc:343] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: e3d8616846c649b6a969b971db0d03db
I20260618 13:59:43.523587 28346 leader_election.cc:304] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
W20260618 13:59:43.523672 28618 leader_election.cc:343] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.523870 28618 leader_election.cc:304] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:43.523867 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc295a8e81b64859915a6335f8fb8d8e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:43.524076 30152 raft_consensus.cc:2804] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:43.524230 30152 raft_consensus.cc:697] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:43.524355 30719 raft_consensus.cc:2749] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.524554 30152 consensus_queue.cc:237] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.524641 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc295a8e81b64859915a6335f8fb8d8e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:43.526682 28484 leader_election.cc:304] T fc295a8e81b64859915a6335f8fb8d8e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.526938 30552 raft_consensus.cc:2749] T fc295a8e81b64859915a6335f8fb8d8e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.527257 30622 raft_consensus.cc:493] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.527302 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 258) took 2433 ms. Trace:
I20260618 13:59:43.527422 30622 raft_consensus.cc:515] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.527393 29746 rpcz_store.cc:276] 0618 13:59:41.093690 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.093917 (+ 227us) service_pool.cc:224] Handling call
0618 13:59:43.527188 (+2433271us) ts_tablet_manager.cc:1151] Deleting tablet 2f3d2f236fd44ab8af0524ea282cfa94
0618 13:59:43.527203 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.527284 (+ 81us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2433153}
W20260618 13:59:43.527937 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 2f3d2f236fd44ab8af0524ea282cfa94 because the tablet was not found. No further retry: Not found: Tablet not found: 2f3d2f236fd44ab8af0524ea282cfa94
I20260618 13:59:43.528122 30622 leader_election.cc:290] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.528277 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 259) took 2434 ms. Trace:
I20260618 13:59:43.528355 29746 rpcz_store.cc:276] 0618 13:59:41.093761 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.095287 (+ 1526us) service_pool.cc:224] Handling call
0618 13:59:43.528191 (+2432904us) ts_tablet_manager.cc:1151] Deleting tablet c557afc3abfa4ef188bf248e89f49d87
0618 13:59:43.528203 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.528261 (+ 58us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2432738}
W20260618 13:59:43.528738 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet c557afc3abfa4ef188bf248e89f49d87 because the tablet was not found. No further retry: Not found: Tablet not found: c557afc3abfa4ef188bf248e89f49d87
I20260618 13:59:43.529560 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 260) took 2427 ms. Trace:
I20260618 13:59:43.533393 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6216474b74a14af6abf977e312b77" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:43.533820 28618 consensus_peers.cc:597] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:43.534333 28346 leader_election.cc:343] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.535694 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6216474b74a14af6abf977e312b77" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:43.529639 29746 rpcz_store.cc:276] 0618 13:59:41.101833 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.103792 (+ 1959us) service_pool.cc:224] Handling call
0618 13:59:43.529485 (+2425693us) ts_tablet_manager.cc:1151] Deleting tablet e6de1293572f4e75af3417f281b9a142
0618 13:59:43.529494 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.529544 (+ 50us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2425514}
W20260618 13:59:43.538749 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet e6de1293572f4e75af3417f281b9a142 because the tablet was not found. No further retry: Not found: Tablet not found: e6de1293572f4e75af3417f281b9a142
I20260618 13:59:43.539132 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 261) took 2254 ms. Trace:
W20260618 13:59:43.539180 28346 leader_election.cc:343] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.539294 28346 leader_election.cc:304] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.539933 30622 raft_consensus.cc:2749] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.540138 29746 rpcz_store.cc:276] 0618 13:59:41.284965 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.285021 (+ 56us) service_pool.cc:224] Handling call
0618 13:59:43.539035 (+2254014us) ts_tablet_manager.cc:1151] Deleting tablet 72c04f55890648f0a8184f3783243f7c
0618 13:59:43.539044 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.539118 (+ 74us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2253839}
W20260618 13:59:43.540532 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 72c04f55890648f0a8184f3783243f7c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72c04f55890648f0a8184f3783243f7c already in progress: creating tablet
I20260618 13:59:43.540889 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 262) took 2246 ms. Trace:
I20260618 13:59:43.543656 29746 rpcz_store.cc:276] 0618 13:59:41.294275 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.294362 (+ 87us) service_pool.cc:224] Handling call
0618 13:59:43.540794 (+2246432us) ts_tablet_manager.cc:1151] Deleting tablet ba2d6894bf5649fc8cfae7e033c432b9
0618 13:59:43.540802 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.540874 (+ 72us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2244422}
W20260618 13:59:43.544303 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet ba2d6894bf5649fc8cfae7e033c432b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba2d6894bf5649fc8cfae7e033c432b9 already in progress: creating tablet
I20260618 13:59:43.544704 29746 tablet_replica.cc:333] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:43.545032 29746 raft_consensus.cc:2243] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:43.545302 29746 raft_consensus.cc:2272] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:43.546371 30622 raft_consensus.cc:493] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.546562 30622 raft_consensus.cc:515] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.547322 30622 leader_election.cc:290] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.548543 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac9f580e35af4d4080d313b0283c0b7e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:43.549480 28346 leader_election.cc:343] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.549950 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac9f580e35af4d4080d313b0283c0b7e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:43.550226 28680 raft_consensus.cc:2468] T ac9f580e35af4d4080d313b0283c0b7e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:43.550936 28346 leader_election.cc:304] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:43.551311 30622 raft_consensus.cc:2804] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:43.551414 30622 raft_consensus.cc:493] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:43.551510 30622 raft_consensus.cc:3060] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.554433 30622 raft_consensus.cc:515] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.555249 30622 leader_election.cc:290] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.555835 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac9f580e35af4d4080d313b0283c0b7e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:43.556633 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac9f580e35af4d4080d313b0283c0b7e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:43.556814 28680 raft_consensus.cc:3060] T ac9f580e35af4d4080d313b0283c0b7e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.559341 28680 raft_consensus.cc:2468] T ac9f580e35af4d4080d313b0283c0b7e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
W20260618 13:59:43.560199 28346 leader_election.cc:343] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.560783 28346 leader_election.cc:304] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:43.561297 30622 raft_consensus.cc:2804] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:43.561432 30622 raft_consensus.cc:697] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:43.561787 30622 consensus_queue.cc:237] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.563843 29746 ts_tablet_manager.cc:1916] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:43.569283 28797 tablet_bootstrap.cc:492] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:43.570461 28797 ts_tablet_manager.cc:1403] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.172s user 0.010s sys 0.030s
I20260618 13:59:43.574373 29746 ts_tablet_manager.cc:1929] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:43.574592 29746 log.cc:1199] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/b0d3d363766546c4a97b0763bad11f71
I20260618 13:59:43.575135 29746 ts_tablet_manager.cc:1950] T b0d3d363766546c4a97b0763bad11f71 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:43.576169 28797 raft_consensus.cc:359] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.576174 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 264) took 2091 ms. Trace:
I20260618 13:59:43.577443 30552 raft_consensus.cc:493] T 8f7b66652f464893ad812d27e883a9a4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.577983 29746 rpcz_store.cc:276] 0618 13:59:41.484956 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.485038 (+ 82us) service_pool.cc:224] Handling call
0618 13:59:43.544646 (+2059608us) ts_tablet_manager.cc:1151] Deleting tablet b0d3d363766546c4a97b0763bad11f71
0618 13:59:43.544658 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.567581 (+ 22923us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.574323 (+ 6742us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.576149 (+ 1826us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2059403}
I20260618 13:59:43.578066 30552 raft_consensus.cc:515] T 8f7b66652f464893ad812d27e883a9a4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.578537 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 265) took 2015 ms. Trace:
I20260618 13:59:43.578611 28797 raft_consensus.cc:385] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.578682 28797 raft_consensus.cc:740] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.578632 29746 rpcz_store.cc:276] 0618 13:59:41.563334 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.563387 (+ 53us) service_pool.cc:224] Handling call
0618 13:59:43.578394 (+2015007us) ts_tablet_manager.cc:1151] Deleting tablet 591465e2b57247a2b224cc8b09225811
0618 13:59:43.578408 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.578518 (+ 110us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2014780}
I20260618 13:59:43.579075 29746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57282 (request call id 266) took 1726 ms. Trace:
I20260618 13:59:43.579165 29746 rpcz_store.cc:276] 0618 13:59:41.852624 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.852677 (+ 53us) service_pool.cc:224] Handling call
0618 13:59:43.578932 (+1726255us) ts_tablet_manager.cc:1151] Deleting tablet 8fea86ca38a741238a59053151087f69
0618 13:59:43.578943 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.579059 (+ 116us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1726056}
I20260618 13:59:43.579331 28797 consensus_queue.cc:260] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.579092 30552 leader_election.cc:290] T 8f7b66652f464893ad812d27e883a9a4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:43.580152 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f7b66652f464893ad812d27e883a9a4" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:43.580468 28797 ts_tablet_manager.cc:1434] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260618 13:59:43.580542 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]) successfully deleted
W20260618 13:59:43.580902 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 591465e2b57247a2b224cc8b09225811 because the tablet was not found. No further retry: Not found: Tablet not found: 591465e2b57247a2b224cc8b09225811
W20260618 13:59:43.581178 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 8fea86ca38a741238a59053151087f69 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8fea86ca38a741238a59053151087f69 already in progress: creating tablet
W20260618 13:59:43.581398 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 already in progress: creating tablet
W20260618 13:59:43.581634 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 2128dbec33db453ba51070767a15b21e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2128dbec33db453ba51070767a15b21e already in progress: creating tablet
W20260618 13:59:43.581887 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet fea86899d2c047d997cebe80551e83c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fea86899d2c047d997cebe80551e83c9 already in progress: creating tablet
I20260618 13:59:43.582114 30622 raft_consensus.cc:493] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.582280 30622 raft_consensus.cc:515] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.583039 30622 leader_election.cc:290] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.584450 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5115a31a38db48c1a6474917aff97c8e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:43.585004 28346 leader_election.cc:343] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:43.598289 28618 consensus_peers.cc:597] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:43.600670 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f7b66652f464893ad812d27e883a9a4" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:43.601099 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5115a31a38db48c1a6474917aff97c8e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:43.601330 28484 leader_election.cc:304] T 8f7b66652f464893ad812d27e883a9a4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.601707 30552 raft_consensus.cc:2749] T 8f7b66652f464893ad812d27e883a9a4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:43.601689 28346 leader_election.cc:343] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.601816 28346 leader_election.cc:304] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.602213 30622 raft_consensus.cc:2749] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.608034 28797 tablet_bootstrap.cc:492] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:43.608282 29566 ts_tablet_manager.cc:1929] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:43.610116 29566 log.cc:1199] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/f45556a59c424ca2bc30956566374c89
I20260618 13:59:43.615036 29566 ts_tablet_manager.cc:1950] T f45556a59c424ca2bc30956566374c89 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:43.617274 28797 tablet_bootstrap.cc:654] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.619168 30152 raft_consensus.cc:493] T e44ad8cb18de42e795da5f229f49cdfe P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.619414 30152 raft_consensus.cc:515] T e44ad8cb18de42e795da5f229f49cdfe P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.617755 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 207) took 8175 ms. Trace:
I20260618 13:59:43.620316 29566 rpcz_store.cc:276] 0618 13:59:35.441894 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.441950 (+ 56us) service_pool.cc:224] Handling call
0618 13:59:43.471689 (+8029739us) ts_tablet_manager.cc:1151] Deleting tablet f45556a59c424ca2bc30956566374c89
0618 13:59:43.471700 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.530494 (+ 58794us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.608216 (+ 77722us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:43.617737 (+ 9521us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8029541}
I20260618 13:59:43.620645 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44ad8cb18de42e795da5f229f49cdfe" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:43.620807 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet f45556a59c424ca2bc30956566374c89 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]) successfully deleted
I20260618 13:59:43.620934 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 209) took 8160 ms. Trace:
I20260618 13:59:43.621025 29566 rpcz_store.cc:276] 0618 13:59:35.460878 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.460925 (+ 47us) service_pool.cc:224] Handling call
0618 13:59:43.620836 (+8159911us) ts_tablet_manager.cc:1151] Deleting tablet eead369de58f40378254697cbbbc6958
0618 13:59:43.620848 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.620916 (+ 68us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8159722}
W20260618 13:59:43.621413 28346 leader_election.cc:343] T e44ad8cb18de42e795da5f229f49cdfe P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.621481 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 210) took 8160 ms. Trace:
W20260618 13:59:43.621508 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet eead369de58f40378254697cbbbc6958 because the tablet was not found. No further retry: Not found: Tablet not found: eead369de58f40378254697cbbbc6958
I20260618 13:59:43.621546 29566 rpcz_store.cc:276] 0618 13:59:35.461512 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.461558 (+ 46us) service_pool.cc:224] Handling call
0618 13:59:43.621395 (+8159837us) ts_tablet_manager.cc:1151] Deleting tablet 5c75278db2234b11bf95f1cf5cc76edd
0618 13:59:43.621405 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.621466 (+ 61us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8159686}
W20260618 13:59:43.621973 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 5c75278db2234b11bf95f1cf5cc76edd because the tablet was not found. No further retry: Not found: Tablet not found: 5c75278db2234b11bf95f1cf5cc76edd
I20260618 13:59:43.622246 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44ad8cb18de42e795da5f229f49cdfe" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:43.622726 28346 leader_election.cc:343] T e44ad8cb18de42e795da5f229f49cdfe P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.622843 28346 leader_election.cc:304] T e44ad8cb18de42e795da5f229f49cdfe P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.623080 30622 raft_consensus.cc:2749] T e44ad8cb18de42e795da5f229f49cdfe P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.623081 30152 leader_election.cc:290] T e44ad8cb18de42e795da5f229f49cdfe P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.633728 29566 tablet_replica.cc:333] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:43.634101 29566 raft_consensus.cc:2243] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:43.634320 29566 raft_consensus.cc:2272] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:43.634981 29566 ts_tablet_manager.cc:1916] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:43.635068 28798 tablet_bootstrap.cc:492] T f2b21dcf74fa4264947ca43cc0026e76 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:43.632027 28799 tablet_bootstrap.cc:492] T 66ada05b01b2427da182998a557529ca P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:43.635207 28798 ts_tablet_manager.cc:1403] T f2b21dcf74fa4264947ca43cc0026e76 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.142s user 0.035s sys 0.048s
I20260618 13:59:43.635229 28799 ts_tablet_manager.cc:1403] T 66ada05b01b2427da182998a557529ca P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.142s user 0.012s sys 0.012s
I20260618 13:59:43.635869 28798 raft_consensus.cc:359] T f2b21dcf74fa4264947ca43cc0026e76 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.636025 28798 raft_consensus.cc:385] T f2b21dcf74fa4264947ca43cc0026e76 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.636072 28798 raft_consensus.cc:740] T f2b21dcf74fa4264947ca43cc0026e76 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.636194 28799 raft_consensus.cc:359] T 66ada05b01b2427da182998a557529ca P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.636415 28799 raft_consensus.cc:385] T 66ada05b01b2427da182998a557529ca P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.636507 28799 raft_consensus.cc:740] T 66ada05b01b2427da182998a557529ca P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.639014 28798 consensus_queue.cc:260] T f2b21dcf74fa4264947ca43cc0026e76 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.639495 28798 ts_tablet_manager.cc:1434] T f2b21dcf74fa4264947ca43cc0026e76 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 13:59:43.640296 28798 tablet_bootstrap.cc:492] T d15b1b603f024f3295eb07c98991076d P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:43.642241 28798 tablet_bootstrap.cc:654] T d15b1b603f024f3295eb07c98991076d P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.645967 28799 consensus_queue.cc:260] T 66ada05b01b2427da182998a557529ca P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.646816 28799 ts_tablet_manager.cc:1434] T 66ada05b01b2427da182998a557529ca P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260618 13:59:43.647282 28799 tablet_bootstrap.cc:492] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:43.649832 30572 consensus_queue.cc:1048] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:43.651636 28799 tablet_bootstrap.cc:654] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.652977 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:43.653496 29020 catalog_manager.cc:5696] T d0bcff62290c43c9be7cc496fbff6142 P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:43.659688 29020 catalog_manager.cc:5696] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:43.664104 28596 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:43.665653 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:43.667577 29020 catalog_manager.cc:5696] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:43.668179 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 8af2bec820b842b4bf6b57482007bd6a (table default.Table_84be6926f4b64876b343577b85db3243 [id=af1859c644b44c1d831314d27a494f94]). Expected version 1 got 0
I20260618 13:59:43.668663 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 9ead386562eb4fd787c7aee776c2194f (table default.Table_93fe53cac2f54d73b1e66cf1661ca928 [id=333c0fd3837b4d89a9197eb6a826c311]). Expected version 3 got 2
I20260618 13:59:43.668814 30622 raft_consensus.cc:493] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.668972 30622 raft_consensus.cc:515] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.669291 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:43.669516 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:43.669656 30622 leader_election.cc:290] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.669708 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:43.670019 28799 tablet_bootstrap.cc:492] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:43.670135 28799 ts_tablet_manager.cc:1403] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.000s
I20260618 13:59:43.670873 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b44c3b0c152442a78edc4736da2cb15c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:43.671509 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b44c3b0c152442a78edc4736da2cb15c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:43.673223 28346 leader_election.cc:343] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:43.673592 28346 leader_election.cc:343] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.673703 28346 leader_election.cc:304] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.674242 30262 raft_consensus.cc:2749] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.676874 28797 tablet_bootstrap.cc:492] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:43.677111 28797 ts_tablet_manager.cc:1403] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.073s user 0.002s sys 0.001s
I20260618 13:59:43.677822 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:43.677843 28797 raft_consensus.cc:359] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.677984 28797 raft_consensus.cc:385] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.679104 28799 raft_consensus.cc:359] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.679338 28799 raft_consensus.cc:385] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.679436 28799 raft_consensus.cc:740] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.684057 28799 consensus_queue.cc:260] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.684733 28799 ts_tablet_manager.cc:1434] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260618 13:59:43.685132 28797 raft_consensus.cc:740] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.685427 28797 consensus_queue.cc:260] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.686062 28797 ts_tablet_manager.cc:1434] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260618 13:59:43.686362 28797 tablet_bootstrap.cc:492] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:43.686554 28618 consensus_peers.cc:597] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:43.686638 28799 tablet_bootstrap.cc:492] T 43853b8aba3444c3a21f6416f224e0d6 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:43.687165 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:43.687968 28797 tablet_bootstrap.cc:654] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.688431 28799 tablet_bootstrap.cc:654] T 43853b8aba3444c3a21f6416f224e0d6 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.688575 29020 catalog_manager.cc:5696] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:43.689008 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_01e1e5741bb54f78996857f6b79fc5d7 [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 2 got 0
I20260618 13:59:43.689352 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet e196dd3c7fe14524a50152643ddbc2ad (table default.Table_2dfebd62a146446f999f1a7a1bd1af79 [id=02ffed3d852845d6845fb5176f036fce]). Expected version 1 got 0
I20260618 13:59:43.689734 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:43.690448 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:43.697064 28458 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:43.702477 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c9ceb3af08fa4618b81cf6f89d7bd0df (table default.Table_21242eba3cb64c7c91082e296439cfe2 [id=10a4f5ef579746b68d8dd37d83f14c3e]). Expected version 1 got 0
I20260618 13:59:43.703147 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:43.705025 28799 tablet_bootstrap.cc:492] T 43853b8aba3444c3a21f6416f224e0d6 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:43.705204 28799 ts_tablet_manager.cc:1403] T 43853b8aba3444c3a21f6416f224e0d6 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.002s
I20260618 13:59:43.705336 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:43.705667 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:43.705890 28799 raft_consensus.cc:359] T 43853b8aba3444c3a21f6416f224e0d6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.706139 28799 raft_consensus.cc:385] T 43853b8aba3444c3a21f6416f224e0d6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.706229 28799 raft_consensus.cc:740] T 43853b8aba3444c3a21f6416f224e0d6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.706522 28799 consensus_queue.cc:260] T 43853b8aba3444c3a21f6416f224e0d6 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.706707 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:43.707182 28799 ts_tablet_manager.cc:1434] T 43853b8aba3444c3a21f6416f224e0d6 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:43.707456 28799 tablet_bootstrap.cc:492] T 4da3d6fa934047c09b3815c4bef511e1 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:43.708750 28799 tablet_bootstrap.cc:654] T 4da3d6fa934047c09b3815c4bef511e1 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.709589 28797 tablet_bootstrap.cc:492] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:43.709776 28797 ts_tablet_manager.cc:1403] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.003s
W20260618 13:59:43.710407 28618 consensus_peers.cc:597] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:43.710734 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:43.710832 28797 raft_consensus.cc:359] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.711048 28797 raft_consensus.cc:385] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.711131 28797 raft_consensus.cc:740] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
W20260618 13:59:43.711350 28990 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet 8af2bec820b842b4bf6b57482007bd6a (table default.Table_84be6926f4b64876b343577b85db3243 [id=af1859c644b44c1d831314d27a494f94], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 8af2bec820b842b4bf6b57482007bd6a consensus state has no leader
I20260618 13:59:43.711418 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_a7356ad0d2d04b9788d4d980093d0df4 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 1 got 0
I20260618 13:59:43.711655 28797 consensus_queue.cc:260] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.712097 28797 ts_tablet_manager.cc:1434] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:43.712332 28797 tablet_bootstrap.cc:492] T ff8e487fd51f4aa18bf59e5f8e011dcd P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:43.713704 28797 tablet_bootstrap.cc:654] T ff8e487fd51f4aa18bf59e5f8e011dcd P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.724069 28797 tablet_bootstrap.cc:492] T ff8e487fd51f4aa18bf59e5f8e011dcd P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:43.724260 28797 ts_tablet_manager.cc:1403] T ff8e487fd51f4aa18bf59e5f8e011dcd P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.004s
I20260618 13:59:43.724877 28797 raft_consensus.cc:359] T ff8e487fd51f4aa18bf59e5f8e011dcd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.725106 28797 raft_consensus.cc:385] T ff8e487fd51f4aa18bf59e5f8e011dcd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.725199 28797 raft_consensus.cc:740] T ff8e487fd51f4aa18bf59e5f8e011dcd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.725555 28797 consensus_queue.cc:260] T ff8e487fd51f4aa18bf59e5f8e011dcd P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.726388 28797 ts_tablet_manager.cc:1434] T ff8e487fd51f4aa18bf59e5f8e011dcd P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:43.726946 28797 tablet_bootstrap.cc:492] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:43.727980 28799 tablet_bootstrap.cc:492] T 4da3d6fa934047c09b3815c4bef511e1 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:43.728163 28799 ts_tablet_manager.cc:1403] T 4da3d6fa934047c09b3815c4bef511e1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.005s
I20260618 13:59:43.728529 28797 tablet_bootstrap.cc:654] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.728693 28540 raft_consensus.cc:3060] T 85b2f34bc17b4a27a91d6cafa9a96c48 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.728925 28799 raft_consensus.cc:359] T 4da3d6fa934047c09b3815c4bef511e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.729188 28799 raft_consensus.cc:385] T 4da3d6fa934047c09b3815c4bef511e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.729296 28799 raft_consensus.cc:740] T 4da3d6fa934047c09b3815c4bef511e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.729575 28799 consensus_queue.cc:260] T 4da3d6fa934047c09b3815c4bef511e1 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.730372 28799 ts_tablet_manager.cc:1434] T 4da3d6fa934047c09b3815c4bef511e1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
W20260618 13:59:43.732543 29026 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 10e9a417eecf4d33ac3bd9bd6325f447 consensus state has no leader
I20260618 13:59:43.732735 29026 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 136) took 2389 ms. Trace:
I20260618 13:59:43.732867 29026 rpcz_store.cc:276] 0618 13:59:41.343614 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:43.179018 (+1835404us) service_pool.cc:224] Handling call
0618 13:59:43.195056 (+ 16038us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb4686956 00007fcbab0e4146 00007fcbab0e3e75 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:43.195336 (+ 280us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:43.195704 (+ 368us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:43.195930 (+ 226us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:43.196026 (+ 96us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:43.269631 (+ 73605us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:43.732700 (+463069us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:43.196714 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:43.196885 (+ 171us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:43.196894 (+ 9us) write_op.cc:435] Acquired schema lock
0618 13:59:43.196898 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:43.197008 (+ 110us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:43.197022 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:43.197027 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:43.197122 (+ 95us) tablet.cc:701] Row locks acquired
0618 13:59:43.197127 (+ 5us) write_op.cc:260] PREPARE: finished
0618 13:59:43.197175 (+ 48us) write_op.cc:270] Start()
0618 13:59:43.197225 (+ 50us) write_op.cc:276] Timestamp: P: 1781791183197124 usec, L: 0
0618 13:59:43.197231 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:43.197428 (+ 197us) log.cc:844] Serialized 405 byte log entry
0618 13:59:43.268488 (+ 71060us) op_driver.cc:464] REPLICATION: finished
0618 13:59:43.268668 (+ 180us) write_op.cc:301] APPLY: starting
0618 13:59:43.268687 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:43.268740 (+ 53us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:43.268805 (+ 65us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:43.268897 (+ 92us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:43.268934 (+ 37us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:43.268943 (+ 9us) write_op.cc:312] APPLY: finished
0618 13:59:43.269075 (+ 132us) log.cc:844] Serialized 23 byte log entry
0618 13:59:43.269176 (+ 101us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:43.269198 (+ 22us) write_op.cc:454] Released schema lock
0618 13:59:43.269207 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:43.269289 (+ 82us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:43.269293 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":36244864,"child_traces":[["op",{"apply.queue_time_us":109,"apply.run_cpu_time_us":616,"apply.run_wall_time_us":616,"num_ops":1,"prepare.queue_time_us":66,"prepare.run_cpu_time_us":846,"prepare.run_wall_time_us":847,"raft.queue_time_us":443,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":8,"replication_time_us":71218}]]}
I20260618 13:59:43.737073 28797 tablet_bootstrap.cc:492] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:43.737195 28797 ts_tablet_manager.cc:1403] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.005s
I20260618 13:59:43.737766 28797 raft_consensus.cc:359] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.739213 28797 raft_consensus.cc:385] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.739408 28797 raft_consensus.cc:740] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.739734 28797 consensus_queue.cc:260] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.740744 28797 ts_tablet_manager.cc:1434] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260618 13:59:43.741366 28797 tablet_bootstrap.cc:492] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:43.741472 28799 tablet_bootstrap.cc:492] T 134a43259d5b429fbe47d0555d9942f3 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:43.743002 28797 tablet_bootstrap.cc:654] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.743669 28799 tablet_bootstrap.cc:654] T 134a43259d5b429fbe47d0555d9942f3 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.747696 29007 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_d334a203a2784021b5e7998a87a91a50 [id=5e62ed351f73400ab201c1b5dd43f28f] alter complete (version 2)
I20260618 13:59:43.751916 28797 tablet_bootstrap.cc:492] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:43.752416 30552 raft_consensus.cc:493] T 5b9f9649587b4c31951204b8bebe7ed8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.752626 28797 ts_tablet_manager.cc:1403] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260618 13:59:43.752604 30552 raft_consensus.cc:515] T 5b9f9649587b4c31951204b8bebe7ed8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.752908 28799 tablet_bootstrap.cc:492] T 134a43259d5b429fbe47d0555d9942f3 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:43.753031 28799 ts_tablet_manager.cc:1403] T 134a43259d5b429fbe47d0555d9942f3 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.001s
I20260618 13:59:43.753232 28797 raft_consensus.cc:359] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.753368 28797 raft_consensus.cc:385] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.753417 28797 raft_consensus.cc:740] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.753415 30552 leader_election.cc:290] T 5b9f9649587b4c31951204b8bebe7ed8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:43.753654 28799 raft_consensus.cc:359] T 134a43259d5b429fbe47d0555d9942f3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.753849 28799 raft_consensus.cc:385] T 134a43259d5b429fbe47d0555d9942f3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.753908 28799 raft_consensus.cc:740] T 134a43259d5b429fbe47d0555d9942f3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.754206 28799 consensus_queue.cc:260] T 134a43259d5b429fbe47d0555d9942f3 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.754351 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9f9649587b4c31951204b8bebe7ed8" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:43.754657 28797 consensus_queue.cc:260] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.755106 28797 ts_tablet_manager.cc:1434] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:43.755111 28799 ts_tablet_manager.cc:1434] T 134a43259d5b429fbe47d0555d9942f3 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:43.755362 28799 tablet_bootstrap.cc:492] T 6827859dd7be4464a9d98ce502df8539 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:43.755362 28797 tablet_bootstrap.cc:492] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:43.756640 30584 raft_consensus.cc:493] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.756886 28799 tablet_bootstrap.cc:654] T 6827859dd7be4464a9d98ce502df8539 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.756791 30584 raft_consensus.cc:515] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.757509 30584 leader_election.cc:290] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:43.758752 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e493f647a344cd581515a77230d765f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:43.756690 28797 tablet_bootstrap.cc:654] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:43.759322 28618 leader_election.cc:343] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.759754 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b9f9649587b4c31951204b8bebe7ed8" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:43.760685 28483 leader_election.cc:304] T 5b9f9649587b4c31951204b8bebe7ed8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.761165 30552 raft_consensus.cc:2749] T 5b9f9649587b4c31951204b8bebe7ed8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.763460 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e493f647a344cd581515a77230d765f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:43.764014 29023 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_330e421310fd41adba536e77619ab0ad [id=abeeb5e876424a1cb99b2a09083c8230] alter complete (version 2)
I20260618 13:59:43.763712 28799 tablet_bootstrap.cc:492] T 6827859dd7be4464a9d98ce502df8539 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:43.764865 28411 raft_consensus.cc:2468] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:43.764889 28799 ts_tablet_manager.cc:1403] T 6827859dd7be4464a9d98ce502df8539 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.002s
I20260618 13:59:43.765268 28617 leader_election.cc:304] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:43.765662 28799 raft_consensus.cc:359] T 6827859dd7be4464a9d98ce502df8539 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.765856 28799 raft_consensus.cc:385] T 6827859dd7be4464a9d98ce502df8539 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260618 13:59:43.765832 28618 consensus_peers.cc:597] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b0d3d363766546c4a97b0763bad11f71. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:43.765905 28799 raft_consensus.cc:740] T 6827859dd7be4464a9d98ce502df8539 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.766012 30584 raft_consensus.cc:2804] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:43.766088 30584 raft_consensus.cc:493] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:43.766165 30584 raft_consensus.cc:3060] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.766145 28799 consensus_queue.cc:260] T 6827859dd7be4464a9d98ce502df8539 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.766515 28797 tablet_bootstrap.cc:492] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:43.766624 28797 ts_tablet_manager.cc:1403] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.002s
I20260618 13:59:43.766885 28799 ts_tablet_manager.cc:1434] T 6827859dd7be4464a9d98ce502df8539 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:43.767156 28799 tablet_bootstrap.cc:492] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:43.767196 28797 raft_consensus.cc:359] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.767326 28797 raft_consensus.cc:385] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.767370 28797 raft_consensus.cc:740] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.767550 28797 consensus_queue.cc:260] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.768026 28797 ts_tablet_manager.cc:1434] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 13:59:43.768282 28797 tablet_bootstrap.cc:492] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:43.768738 28799 tablet_bootstrap.cc:654] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.769688 28797 tablet_bootstrap.cc:654] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.776640 30622 raft_consensus.cc:493] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.776791 30622 raft_consensus.cc:515] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.777424 30622 leader_election.cc:290] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.778939 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2a6fabad00642528913e735929e295e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:43.779587 28346 leader_election.cc:343] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.779834 30584 raft_consensus.cc:515] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.780376 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2a6fabad00642528913e735929e295e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:43.781149 28346 leader_election.cc:343] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.781270 28346 leader_election.cc:304] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.781514 30622 raft_consensus.cc:2749] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.781863 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e493f647a344cd581515a77230d765f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:43.782377 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e493f647a344cd581515a77230d765f" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:43.782565 28403 raft_consensus.cc:3060] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:43.782593 28618 leader_election.cc:343] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.782915 30584 leader_election.cc:290] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:43.782972 28799 tablet_bootstrap.cc:492] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:43.783080 28799 ts_tablet_manager.cc:1403] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.004s
I20260618 13:59:43.783921 28799 raft_consensus.cc:359] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.784060 28799 raft_consensus.cc:385] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.784107 28799 raft_consensus.cc:740] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.784282 28799 consensus_queue.cc:260] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.784998 28799 ts_tablet_manager.cc:1434] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:43.785091 28403 raft_consensus.cc:2468] T 2e493f647a344cd581515a77230d765f P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:43.785251 28799 tablet_bootstrap.cc:492] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:43.785488 28617 leader_election.cc:304] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:43.786157 30584 raft_consensus.cc:2804] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:43.787477 30584 raft_consensus.cc:697] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:43.788054 30584 consensus_queue.cc:237] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.790669 29011 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_8f61f6724b1d44408a1d2efa16fe8a2d [id=cf981768efab4f4c9cc5353ac9c053a6] alter complete (version 1)
I20260618 13:59:43.792899 28799 tablet_bootstrap.cc:654] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.795524 28797 tablet_bootstrap.cc:492] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:43.796064 28797 ts_tablet_manager.cc:1403] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.000s
I20260618 13:59:43.796917 29024 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet c5d046e0421142cd84c5b12d6a434f53
I20260618 13:59:43.797639 29024 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 133) took 2527 ms. Trace:
I20260618 13:59:43.798090 30844 tablet.cc:1724] T 438663f365e34fab87e34f434920cecf P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:43.797775 29024 rpcz_store.cc:276] 0618 13:59:41.269894 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:43.042558 (+1772664us) service_pool.cc:224] Handling call
0618 13:59:43.043010 (+ 452us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:43.043279 (+ 269us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:43.043334 (+ 55us) catalog_manager.cc:2849] Locking tablets
0618 13:59:43.070699 (+ 27365us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:43.165026 (+ 94327us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:43.165104 (+ 78us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:43.796855 (+631751us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:43.797583 (+ 728us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:43.071470 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:43.071609 (+ 139us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:43.071617 (+ 8us) write_op.cc:435] Acquired schema lock
0618 13:59:43.071619 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:43.071731 (+ 112us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:43.071744 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:43.071749 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:43.071865 (+ 116us) tablet.cc:701] Row locks acquired
0618 13:59:43.071867 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:43.071904 (+ 37us) write_op.cc:270] Start()
0618 13:59:43.071946 (+ 42us) write_op.cc:276] Timestamp: P: 1781791183071873 usec, L: 0
0618 13:59:43.071950 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:43.072113 (+ 163us) log.cc:844] Serialized 768 byte log entry
0618 13:59:43.163557 (+ 91444us) op_driver.cc:464] REPLICATION: finished
0618 13:59:43.163740 (+ 183us) write_op.cc:301] APPLY: starting
0618 13:59:43.163758 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:43.163809 (+ 51us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:43.163839 (+ 30us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:43.163921 (+ 82us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:43.163957 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:43.163964 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:43.164051 (+ 87us) log.cc:844] Serialized 29 byte log entry
0618 13:59:43.164124 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:43.164144 (+ 20us) write_op.cc:454] Released schema lock
0618 13:59:43.164152 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:43.164229 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:43.164236 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":128,"apply.run_cpu_time_us":480,"apply.run_wall_time_us":479,"num_ops":2,"prepare.queue_time_us":51,"prepare.run_cpu_time_us":738,"prepare.run_wall_time_us":739,"raft.queue_time_us":1813,"raft.run_cpu_time_us":8,"raft.run_wall_time_us":9,"replication_time_us":91576}]]}
I20260618 13:59:43.799732 28797 raft_consensus.cc:359] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.800151 28797 raft_consensus.cc:385] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.800302 28797 raft_consensus.cc:740] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.800985 28797 consensus_queue.cc:260] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.801993 28799 tablet_bootstrap.cc:492] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:43.802132 28799 ts_tablet_manager.cc:1403] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.001s
I20260618 13:59:43.802600 28797 ts_tablet_manager.cc:1434] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260618 13:59:43.803318 28797 tablet_bootstrap.cc:492] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:43.803283 28799 raft_consensus.cc:359] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:43.803471 28799 raft_consensus.cc:385] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.803529 28799 raft_consensus.cc:740] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.803725 28799 consensus_queue.cc:260] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:43.804768 28799 ts_tablet_manager.cc:1434] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:43.805377 28644 tablet_service.cc:1558] Processing DeleteTablet for tablet c5d046e0421142cd84c5b12d6a434f53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:43 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:43.805393 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet c5d046e0421142cd84c5b12d6a434f53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:43 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:43.805547 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet c5d046e0421142cd84c5b12d6a434f53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:43 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:43.805655 29746 tablet_replica.cc:333] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:43.805778 30777 tablet_replica.cc:333] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:43.805876 29746 raft_consensus.cc:2243] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:43.805990 30777 raft_consensus.cc:2243] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:43.806074 29746 raft_consensus.cc:2272] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:43.806169 30777 raft_consensus.cc:2272] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:43.806743 30777 ts_tablet_manager.cc:1916] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:43.806754 29746 ts_tablet_manager.cc:1916] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:43.808141 28797 tablet_bootstrap.cc:654] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.813052 30777 ts_tablet_manager.cc:1929] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:43.813170 30777 log.cc:1199] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/c5d046e0421142cd84c5b12d6a434f53
I20260618 13:59:43.813555 30777 ts_tablet_manager.cc:1950] T c5d046e0421142cd84c5b12d6a434f53 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:43.814850 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]) successfully deleted
I20260618 13:59:43.815501 28797 tablet_bootstrap.cc:492] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:43.816011 28797 ts_tablet_manager.cc:1403] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260618 13:59:43.816825 28797 raft_consensus.cc:359] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.817745 28797 raft_consensus.cc:385] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.817914 28797 raft_consensus.cc:740] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.818317 28797 consensus_queue.cc:260] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.819149 28797 ts_tablet_manager.cc:1434] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260618 13:59:43.819795 28797 tablet_bootstrap.cc:492] T 0e6f3f8431cd49b5bf1306bc7609761d P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:43.821823 30552 raft_consensus.cc:493] T eebebb79db644535b54a48e4dee89d59 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.821954 30552 raft_consensus.cc:515] T eebebb79db644535b54a48e4dee89d59 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.822515 30552 leader_election.cc:290] T eebebb79db644535b54a48e4dee89d59 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:43.825309 29017 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 134) took 2544 ms. Trace:
I20260618 13:59:43.825393 28797 tablet_bootstrap.cc:654] T 0e6f3f8431cd49b5bf1306bc7609761d P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.825860 29303 consensus_queue.cc:1048] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:43.825527 29017 rpcz_store.cc:276] 0618 13:59:41.280566 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:43.044524 (+1763958us) service_pool.cc:224] Handling call
0618 13:59:43.044805 (+ 281us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:43.045000 (+ 195us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:43.045158 (+ 158us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:43.045210 (+ 52us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:43.162682 (+117472us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:43.825261 (+662579us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:43.050492 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:43.050585 (+ 93us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:43.050590 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:43.050593 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:43.050674 (+ 81us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:43.050682 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:43.050687 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:43.050737 (+ 50us) tablet.cc:701] Row locks acquired
0618 13:59:43.050740 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:43.050768 (+ 28us) write_op.cc:270] Start()
0618 13:59:43.050795 (+ 27us) write_op.cc:276] Timestamp: P: 1781791183050741 usec, L: 0
0618 13:59:43.050798 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:43.050973 (+ 175us) log.cc:844] Serialized 405 byte log entry
0618 13:59:43.121216 (+ 70243us) op_driver.cc:464] REPLICATION: finished
0618 13:59:43.131234 (+ 10018us) write_op.cc:301] APPLY: starting
0618 13:59:43.131324 (+ 90us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:43.131368 (+ 44us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:43.133845 (+ 2477us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:43.133913 (+ 68us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:43.133943 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:43.133949 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:43.134078 (+ 129us) log.cc:844] Serialized 23 byte log entry
0618 13:59:43.139347 (+ 5269us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:43.139363 (+ 16us) write_op.cc:454] Released schema lock
0618 13:59:43.139370 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:43.162813 (+ 23443us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:43.162821 (+ 8us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":9800,"apply.run_cpu_time_us":570,"apply.run_wall_time_us":31563,"num_ops":1,"prepare.queue_time_us":4887,"prepare.run_cpu_time_us":572,"prepare.run_wall_time_us":572,"raft.queue_time_us":825,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":8,"replication_time_us":70381}]]}
I20260618 13:59:43.837955 29746 ts_tablet_manager.cc:1929] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:43.838168 29746 log.cc:1199] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/c5d046e0421142cd84c5b12d6a434f53
I20260618 13:59:43.838768 29746 ts_tablet_manager.cc:1950] T c5d046e0421142cd84c5b12d6a434f53 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:43.839607 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]) successfully deleted
I20260618 13:59:43.842796 30622 consensus_queue.cc:1048] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:43.867142 30152 raft_consensus.cc:493] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.867394 30152 raft_consensus.cc:515] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.868692 28545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dab5f1faf6bf46e2b7d9c3808511ef5a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:43.869233 28346 leader_election.cc:343] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.868953 30152 leader_election.cc:290] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.887874 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dab5f1faf6bf46e2b7d9c3808511ef5a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:43.890200 28346 leader_election.cc:343] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.890347 28346 leader_election.cc:304] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.891119 30152 raft_consensus.cc:2749] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.895124 28798 tablet_bootstrap.cc:492] T d15b1b603f024f3295eb07c98991076d P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:43.895262 28798 ts_tablet_manager.cc:1403] T d15b1b603f024f3295eb07c98991076d P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.255s user 0.042s sys 0.047s
I20260618 13:59:43.895854 28798 raft_consensus.cc:359] T d15b1b603f024f3295eb07c98991076d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.896010 28798 raft_consensus.cc:385] T d15b1b603f024f3295eb07c98991076d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.896059 28798 raft_consensus.cc:740] T d15b1b603f024f3295eb07c98991076d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.896272 28798 consensus_queue.cc:260] T d15b1b603f024f3295eb07c98991076d P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.896713 28798 ts_tablet_manager.cc:1434] T d15b1b603f024f3295eb07c98991076d P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260618 13:59:43.896952 28798 tablet_bootstrap.cc:492] T 9559c5f2c9e94e858093dd2f0865c865 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:43.898240 28798 tablet_bootstrap.cc:654] T 9559c5f2c9e94e858093dd2f0865c865 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.902312 30152 raft_consensus.cc:493] T c9382a86a2f645b4857aa04e459209d9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.902483 30152 raft_consensus.cc:515] T c9382a86a2f645b4857aa04e459209d9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.903157 30152 leader_election.cc:290] T c9382a86a2f645b4857aa04e459209d9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.904094 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9382a86a2f645b4857aa04e459209d9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:43.904202 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9382a86a2f645b4857aa04e459209d9" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:43.904582 28346 leader_election.cc:343] T c9382a86a2f645b4857aa04e459209d9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:43.904776 28346 leader_election.cc:343] T c9382a86a2f645b4857aa04e459209d9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.904858 28346 leader_election.cc:304] T c9382a86a2f645b4857aa04e459209d9 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:43.905160 30152 raft_consensus.cc:2749] T c9382a86a2f645b4857aa04e459209d9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:43.911408 28798 tablet_bootstrap.cc:492] T 9559c5f2c9e94e858093dd2f0865c865 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:43.911623 28798 ts_tablet_manager.cc:1403] T 9559c5f2c9e94e858093dd2f0865c865 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.006s
I20260618 13:59:43.912372 28798 raft_consensus.cc:359] T 9559c5f2c9e94e858093dd2f0865c865 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.912614 28798 raft_consensus.cc:385] T 9559c5f2c9e94e858093dd2f0865c865 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:43.912716 28798 raft_consensus.cc:740] T 9559c5f2c9e94e858093dd2f0865c865 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:43.912989 28798 consensus_queue.cc:260] T 9559c5f2c9e94e858093dd2f0865c865 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.913481 28798 ts_tablet_manager.cc:1434] T 9559c5f2c9e94e858093dd2f0865c865 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:43.914012 28798 tablet_bootstrap.cc:492] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:43.916329 28798 tablet_bootstrap.cc:654] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:43.925748 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebebb79db644535b54a48e4dee89d59" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:43.936172 30843 tablet.cc:1724] T 438663f365e34fab87e34f434920cecf P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260618 13:59:43.946561 28346 consensus_peers.cc:597] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:43.949895 28618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 124252us
I20260618 13:59:43.950387 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebebb79db644535b54a48e4dee89d59" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:43.951596 28540 raft_consensus.cc:3060] T 5b9f9649587b4c31951204b8bebe7ed8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.953612 30262 raft_consensus.cc:493] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.953745 30262 raft_consensus.cc:515] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.954308 30262 leader_election.cc:290] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.955024 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8e487fd51f4aa18bf59e5f8e011dcd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:43.955487 28346 leader_election.cc:343] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.957644 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8e487fd51f4aa18bf59e5f8e011dcd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:43.957811 28683 raft_consensus.cc:2468] T ff8e487fd51f4aa18bf59e5f8e011dcd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:43.958233 28346 leader_election.cc:304] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:43.958729 30262 raft_consensus.cc:2804] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:43.958808 30262 raft_consensus.cc:493] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:43.958882 30262 raft_consensus.cc:3060] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.961459 30262 raft_consensus.cc:515] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.961995 30262 leader_election.cc:290] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.962909 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8e487fd51f4aa18bf59e5f8e011dcd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:43.963248 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8e487fd51f4aa18bf59e5f8e011dcd" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
W20260618 13:59:43.963375 28346 leader_election.cc:343] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:43.963425 28683 raft_consensus.cc:3060] T ff8e487fd51f4aa18bf59e5f8e011dcd P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:43.966141 28683 raft_consensus.cc:2468] T ff8e487fd51f4aa18bf59e5f8e011dcd P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:43.966814 28346 leader_election.cc:304] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:43.967425 30152 raft_consensus.cc:2804] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:43.967537 30152 raft_consensus.cc:697] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:43.968156 30152 consensus_queue.cc:237] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.978780 29303 raft_consensus.cc:493] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.978907 29303 raft_consensus.cc:515] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.979453 29303 leader_election.cc:290] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:43.994840 30262 raft_consensus.cc:493] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:43.995083 30262 raft_consensus.cc:515] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:43.995747 30262 leader_election.cc:290] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:43.997201 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b91f053bcc42bb829bb8180e232c8e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:43.997852 28346 leader_election.cc:343] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.003458 30864 raft_consensus.cc:493] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.003988 30864 raft_consensus.cc:515] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.005617 30864 leader_election.cc:290] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:44.006003 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b602a473a94c05a617fb9478300c74" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:44.006628 28346 leader_election.cc:343] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.007799 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b602a473a94c05a617fb9478300c74" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:44.008584 28346 leader_election.cc:343] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.008754 28346 leader_election.cc:304] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:44.018718 30152 raft_consensus.cc:2749] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:44.019182 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b91f053bcc42bb829bb8180e232c8e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:44.019829 28346 leader_election.cc:343] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.020213 28346 leader_election.cc:304] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:44.020659 30152 raft_consensus.cc:2749] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:44.020954 30838 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a950ab7cd66b4cb0b29592aa692c2dd0)
I20260618 13:59:44.021215 30838 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" member_type: VOTER last_known_addr { host: "127.27.100.62" port: 43001 } } peers { permanent_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" member_type: VOTER last_known_addr { host: "127.27.100.61" port: 45967 } } peers { permanent_uuid: "7637285570574a6dbcef1de6f84ecac0" member_type: VOTER last_known_addr { host: "127.27.100.60" port: 42083 } }
I20260618 13:59:44.022460 30838 leader_election.cc:290] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a950ab7cd66b4cb0b29592aa692c2dd0 (127.27.100.61:45967), 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083)
I20260618 13:59:44.023197 29048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e4d4f4566be42dfbafb7c0e40fb17ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 673 } } ignore_live_leader: false dest_uuid: "a950ab7cd66b4cb0b29592aa692c2dd0" is_pre_election: true
I20260618 13:59:44.023973 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5836982752b4209b72a372c1dfeb71c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:44.024269 28411 raft_consensus.cc:2468] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:44.030722 28617 leader_election.cc:304] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:44.032819 30847 raft_consensus.cc:2804] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:44.033004 30847 raft_consensus.cc:493] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:44.033164 30847 raft_consensus.cc:3060] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.040181 29566 ts_tablet_manager.cc:1929] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:44.041070 29566 log.cc:1199] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/98446f40214c4ac984fcfe98310005c5
I20260618 13:59:44.041550 29566 ts_tablet_manager.cc:1950] T 98446f40214c4ac984fcfe98310005c5 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:44.042497 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 213) took 8348 ms. Trace:
W20260618 13:59:44.042639 28071 leader_election.cc:336] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083): Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111)
I20260618 13:59:44.042893 28071 leader_election.cc:304] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e4d4f4566be42dfbafb7c0e40fb17ff; no voters: 7637285570574a6dbcef1de6f84ecac0, a950ab7cd66b4cb0b29592aa692c2dd0
I20260618 13:59:44.047582 28540 raft_consensus.cc:3060] T fc295a8e81b64859915a6335f8fb8d8e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.055068 28528 tablet_service.cc:1511] Processing CreateTablet for tablet 7c40fe72e00a447582a35611de8ffa7c (DEFAULT_TABLE table=default.Table_a4350f6ab3f646c881c841735019f707 [id=435a59d5315347b1b2ec73b9eb137e42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:44.055183 28484 leader_election.cc:304] T eebebb79db644535b54a48e4dee89d59 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:44.055469 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c40fe72e00a447582a35611de8ffa7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:44.055553 30552 raft_consensus.cc:2749] T eebebb79db644535b54a48e4dee89d59 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:44.056735 29303 raft_consensus.cc:493] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.056855 29303 raft_consensus.cc:515] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.057396 29303 leader_election.cc:290] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.062845 30838 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 9e4d4f4566be42dfbafb7c0e40fb17ff [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260618 13:59:44.066792 30864 raft_consensus.cc:493] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.066942 30864 raft_consensus.cc:515] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.068192 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1cf014708ea4c4085b37c6094bf8a20" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:44.069725 28346 leader_election.cc:343] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.070209 30152 raft_consensus.cc:493] T 4da3d6fa934047c09b3815c4bef511e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.070474 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 7c40fe72e00a447582a35611de8ffa7c (DEFAULT_TABLE table=default.Table_a4350f6ab3f646c881c841735019f707 [id=435a59d5315347b1b2ec73b9eb137e42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:44.070603 30152 raft_consensus.cc:515] T 4da3d6fa934047c09b3815c4bef511e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.070905 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c40fe72e00a447582a35611de8ffa7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:44.071246 30152 leader_election.cc:290] T 4da3d6fa934047c09b3815c4bef511e1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:44.071843 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da3d6fa934047c09b3815c4bef511e1" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:44.072476 28346 leader_election.cc:343] T 4da3d6fa934047c09b3815c4bef511e1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.072968 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da3d6fa934047c09b3815c4bef511e1" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:44.073498 28346 leader_election.cc:343] T 4da3d6fa934047c09b3815c4bef511e1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.073608 28346 leader_election.cc:304] T 4da3d6fa934047c09b3815c4bef511e1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:44.074071 30864 leader_election.cc:290] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:44.074211 30864 raft_consensus.cc:2749] T 4da3d6fa934047c09b3815c4bef511e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:44.077596 28799 tablet_bootstrap.cc:492] T 7c40fe72e00a447582a35611de8ffa7c P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:44.079785 28799 tablet_bootstrap.cc:654] T 7c40fe72e00a447582a35611de8ffa7c P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.081027 30847 raft_consensus.cc:515] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.083000 28546 raft_consensus.cc:3060] T eebebb79db644535b54a48e4dee89d59 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.089766 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1cf014708ea4c4085b37c6094bf8a20" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:44.090716 28346 leader_election.cc:343] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.090896 28346 leader_election.cc:304] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:44.091740 30152 raft_consensus.cc:2749] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:44.093407 30847 leader_election.cc:290] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.096192 28799 tablet_bootstrap.cc:492] T 7c40fe72e00a447582a35611de8ffa7c P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:44.096421 28799 ts_tablet_manager.cc:1403] T 7c40fe72e00a447582a35611de8ffa7c P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.001s
I20260618 13:59:44.096894 30622 raft_consensus.cc:493] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.097020 30622 raft_consensus.cc:515] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.097554 30622 leader_election.cc:290] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:44.097452 28799 raft_consensus.cc:359] T 7c40fe72e00a447582a35611de8ffa7c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.097654 28799 raft_consensus.cc:385] T 7c40fe72e00a447582a35611de8ffa7c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.097715 28799 raft_consensus.cc:740] T 7c40fe72e00a447582a35611de8ffa7c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.097985 28799 consensus_queue.cc:260] T 7c40fe72e00a447582a35611de8ffa7c P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.098753 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52006500f117417b8a2c979d4aeb077c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:44.098851 28799 ts_tablet_manager.cc:1434] T 7c40fe72e00a447582a35611de8ffa7c P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260618 13:59:44.100620 28346 leader_election.cc:343] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.102108 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52006500f117417b8a2c979d4aeb077c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:44.102514 28680 raft_consensus.cc:2468] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:44.103320 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52006500f117417b8a2c979d4aeb077c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:44.103562 28408 raft_consensus.cc:2468] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:44.104776 28346 leader_election.cc:304] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:44.105310 30864 raft_consensus.cc:2804] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:44.105481 30864 raft_consensus.cc:493] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:44.105634 30864 raft_consensus.cc:3060] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.111838 30864 raft_consensus.cc:515] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.121558 28644 tablet_service.cc:1511] Processing CreateTablet for tablet 7c40fe72e00a447582a35611de8ffa7c (DEFAULT_TABLE table=default.Table_a4350f6ab3f646c881c841735019f707 [id=435a59d5315347b1b2ec73b9eb137e42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:44.121961 28644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c40fe72e00a447582a35611de8ffa7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:44.124534 30552 raft_consensus.cc:493] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.124681 30552 raft_consensus.cc:515] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.125355 30552 leader_election.cc:290] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.127545 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5836982752b4209b72a372c1dfeb71c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:44.127666 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:44.132479 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c9ceb3af08fa4618b81cf6f89d7bd0df (table default.Table_21242eba3cb64c7c91082e296439cfe2 [id=10a4f5ef579746b68d8dd37d83f14c3e]). Expected version 1 got 0
W20260618 13:59:44.132709 28617 consensus_peers.cc:597] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c5d046e0421142cd84c5b12d6a434f53. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:44.133064 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:44.137451 30847 consensus_queue.cc:1048] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:44.138167 28617 leader_election.cc:304] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:44.138454 30847 raft_consensus.cc:2804] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:44.138535 30847 raft_consensus.cc:493] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:44.138617 30847 raft_consensus.cc:3060] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.141413 30847 raft_consensus.cc:515] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.142007 30847 leader_election.cc:290] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.142275 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_c61ea74d5c6d4a718da2ab92f4a52397 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 2 got 0
I20260618 13:59:44.142620 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet e196dd3c7fe14524a50152643ddbc2ad (table default.Table_2dfebd62a146446f999f1a7a1bd1af79 [id=02ffed3d852845d6845fb5176f036fce]). Expected version 1 got 0
I20260618 13:59:44.042913 29566 rpcz_store.cc:276] 0618 13:59:35.694496 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:35.698754 (+ 4258us) service_pool.cc:224] Handling call
0618 13:59:43.631180 (+7932426us) ts_tablet_manager.cc:1151] Deleting tablet 98446f40214c4ac984fcfe98310005c5
0618 13:59:43.633677 (+ 2497us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:43.744117 (+110440us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:44.040119 (+296002us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:44.042477 (+ 2358us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7931497}
I20260618 13:59:44.143343 29566 tablet_replica.cc:333] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:44.143558 29566 raft_consensus.cc:2243] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [term 2 LEADER]: Raft consensus shutting down.
I20260618 13:59:44.143567 28797 tablet_bootstrap.cc:492] T 0e6f3f8431cd49b5bf1306bc7609761d P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:44.143767 28797 ts_tablet_manager.cc:1403] T 0e6f3f8431cd49b5bf1306bc7609761d P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.324s user 0.030s sys 0.087s
I20260618 13:59:44.143939 29566 pending_rounds.cc:70] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257: Trying to abort 1 pending ops.
I20260618 13:59:44.143992 29566 pending_rounds.cc:77] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257: Aborting op as it isn't in flight: id { term: 2 index: 3 } timestamp: 7298216679534989312 op_type: NO_OP noop_request { }
I20260618 13:59:44.144088 29566 raft_consensus.cc:2889] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:44.144169 29566 raft_consensus.cc:2272] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:44.143778 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:44.144336 28797 raft_consensus.cc:359] T 0e6f3f8431cd49b5bf1306bc7609761d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.144479 28797 raft_consensus.cc:385] T 0e6f3f8431cd49b5bf1306bc7609761d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.144593 28797 raft_consensus.cc:740] T 0e6f3f8431cd49b5bf1306bc7609761d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.144732 29566 ts_tablet_manager.cc:1916] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:44.144760 28797 consensus_queue.cc:260] T 0e6f3f8431cd49b5bf1306bc7609761d P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.145113 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:44.145330 28797 ts_tablet_manager.cc:1434] T 0e6f3f8431cd49b5bf1306bc7609761d P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260618 13:59:44.145576 28797 tablet_bootstrap.cc:492] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:44.147898 28797 tablet_bootstrap.cc:654] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.148976 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52006500f117417b8a2c979d4aeb077c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:44.149291 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 10e9a417eecf4d33ac3bd9bd6325f447 (table default.Table_21426dbf385d47d2af5fc25e287ed0ed [id=0de5461053eb4517b15d6b91875fc04d]). Expected version 1 got 0
I20260618 13:59:44.150473 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5836982752b4209b72a372c1dfeb71c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:44.151634 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:44.152390 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:44.152776 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:44.155424 28546 raft_consensus.cc:3060] T f2b21dcf74fa4264947ca43cc0026e76 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.157449 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:44.157752 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 00b602a473a94c05a617fb9478300c74 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:44.163501 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_01e1e5741bb54f78996857f6b79fc5d7 [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 2 got 0
I20260618 13:59:44.163546 28797 tablet_bootstrap.cc:492] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:44.163693 28797 ts_tablet_manager.cc:1403] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.005s
I20260618 13:59:44.164505 28797 raft_consensus.cc:359] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.164712 28797 raft_consensus.cc:385] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.164784 28797 raft_consensus.cc:740] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.165033 28797 consensus_queue.cc:260] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.165652 28797 ts_tablet_manager.cc:1434] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:44.165994 28797 tablet_bootstrap.cc:492] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:44.166632 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:44.167795 28797 tablet_bootstrap.cc:654] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.168200 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:44.173141 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:44.173513 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 8af2bec820b842b4bf6b57482007bd6a (table default.Table_84be6926f4b64876b343577b85db3243 [id=af1859c644b44c1d831314d27a494f94]). Expected version 1 got 0
I20260618 13:59:44.174999 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:44.175264 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:44.178539 30552 raft_consensus.cc:493] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.178661 30552 raft_consensus.cc:515] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.179177 30552 leader_election.cc:290] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:44.180056 28483 leader_election.cc:343] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: c01c8463055e4526b3c75190066f30b6
I20260618 13:59:44.182813 29024 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:44.185549 28798 tablet_bootstrap.cc:492] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:44.185819 28798 ts_tablet_manager.cc:1403] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.272s user 0.032s sys 0.060s
I20260618 13:59:44.186808 28798 raft_consensus.cc:359] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:44.187240 28798 raft_consensus.cc:385] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.187347 28798 raft_consensus.cc:740] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.187743 28798 consensus_queue.cc:260] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:44.188547 28798 ts_tablet_manager.cc:1434] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:44.188934 28798 tablet_bootstrap.cc:492] T 8af2bec820b842b4bf6b57482007bd6a P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:44.190701 28798 tablet_bootstrap.cc:654] T 8af2bec820b842b4bf6b57482007bd6a P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.193166 30880 tablet.cc:1724] T 438663f365e34fab87e34f434920cecf P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:44.199409 28730 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:44.199651 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed648dcf882340388155068d7a5a7869" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:44.206560 30864 leader_election.cc:290] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:44.126155 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed648dcf882340388155068d7a5a7869" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:44.160221 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5836982752b4209b72a372c1dfeb71c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:44.207422 28411 raft_consensus.cc:3060] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.207594 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52006500f117417b8a2c979d4aeb077c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:44.207808 28680 raft_consensus.cc:2393] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fc808af0e884ed8836561665d8e0e7b in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:44.208434 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52006500f117417b8a2c979d4aeb077c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:44.208976 28346 leader_election.cc:343] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.209072 28346 leader_election.cc:304] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:44.209604 28797 tablet_bootstrap.cc:492] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:44.209776 28797 ts_tablet_manager.cc:1403] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.044s user 0.008s sys 0.003s
I20260618 13:59:44.209828 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52006500f117417b8a2c979d4aeb077c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:44.210009 28408 raft_consensus.cc:2393] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d6a169b16ced46ca9cfbc631709bb257 in current term 1: Already voted for candidate 6fc808af0e884ed8836561665d8e0e7b in this term.
I20260618 13:59:44.210701 28797 raft_consensus.cc:359] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.210865 28797 raft_consensus.cc:385] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.210918 28797 raft_consensus.cc:740] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.211076 28797 consensus_queue.cc:260] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.211169 28411 raft_consensus.cc:2468] T c5836982752b4209b72a372c1dfeb71c P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:44.211516 28797 ts_tablet_manager.cc:1434] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:44.211761 28797 tablet_bootstrap.cc:492] T d3ed5c1137434fcc80ed0ca56a323be7 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:44.212160 30856 raft_consensus.cc:493] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.212317 30856 raft_consensus.cc:515] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.212900 30856 leader_election.cc:290] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.213307 28797 tablet_bootstrap.cc:654] T d3ed5c1137434fcc80ed0ca56a323be7 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.213649 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01c8463055e4526b3c75190066f30b6" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:44.213865 28680 raft_consensus.cc:2468] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:44.219630 30864 raft_consensus.cc:2749] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260618 13:59:44.223811 28548 raft_consensus.cc:3060] T d15b1b603f024f3295eb07c98991076d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.232272 29303 raft_consensus.cc:493] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.232506 29303 raft_consensus.cc:515] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.233088 29303 leader_election.cc:290] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.249235 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9ceb3af08fa4618b81cf6f89d7bd0df" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:44.249611 28407 raft_consensus.cc:2468] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:44.250315 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52006500f117417b8a2c979d4aeb077c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:44.265226 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:44.268829 29566 ts_tablet_manager.cc:1929] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.3
I20260618 13:59:44.269264 29566 log.cc:1199] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/cea4bc13eabc4dd98b4f96723498014c
I20260618 13:59:44.269855 29566 ts_tablet_manager.cc:1950] T cea4bc13eabc4dd98b4f96723498014c P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:44.270730 28798 tablet_bootstrap.cc:492] T 8af2bec820b842b4bf6b57482007bd6a P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:44.270998 28798 ts_tablet_manager.cc:1403] T 8af2bec820b842b4bf6b57482007bd6a P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.082s user 0.010s sys 0.007s
I20260618 13:59:44.271399 30152 raft_consensus.cc:493] T 25463905e343470d941f89a53efbc2d1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.271549 30152 raft_consensus.cc:515] T 25463905e343470d941f89a53efbc2d1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.271755 28798 raft_consensus.cc:359] T 8af2bec820b842b4bf6b57482007bd6a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.272034 28798 raft_consensus.cc:385] T 8af2bec820b842b4bf6b57482007bd6a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.272137 28798 raft_consensus.cc:740] T 8af2bec820b842b4bf6b57482007bd6a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.272393 30152 leader_election.cc:290] T 25463905e343470d941f89a53efbc2d1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:44.272387 28798 consensus_queue.cc:260] T 8af2bec820b842b4bf6b57482007bd6a P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.273721 28798 ts_tablet_manager.cc:1434] T 8af2bec820b842b4bf6b57482007bd6a P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260618 13:59:44.274698 28798 tablet_bootstrap.cc:492] T f695a7e6a925470787a708c365e6b86b P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:44.276762 28798 tablet_bootstrap.cc:654] T f695a7e6a925470787a708c365e6b86b P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.277828 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25463905e343470d941f89a53efbc2d1" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:44.281791 28537 raft_consensus.cc:3060] T 9559c5f2c9e94e858093dd2f0865c865 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.284013 29026 catalog_manager.cc:5696] T a3c74d3ca66f42ad83ef54d9459b99fd P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:44.277622 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25463905e343470d941f89a53efbc2d1" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:44.285902 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 225) took 6874 ms. Trace:
I20260618 13:59:44.286005 29566 rpcz_store.cc:276] 0618 13:59:37.411142 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:37.411202 (+ 60us) service_pool.cc:224] Handling call
0618 13:59:44.143295 (+6732093us) ts_tablet_manager.cc:1151] Deleting tablet cea4bc13eabc4dd98b4f96723498014c
0618 13:59:44.143305 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.249322 (+106017us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:44.268768 (+ 19446us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:44.285884 (+ 17116us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6731848}
I20260618 13:59:44.289362 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet cea4bc13eabc4dd98b4f96723498014c (table default.Table_e372385b751446e28fe8798e387b2148 [id=38a8a1053c4c41d484c0b5bc4439dc0c]) successfully deleted
W20260618 13:59:44.290174 28346 leader_election.cc:343] T 25463905e343470d941f89a53efbc2d1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:44.290773 28346 leader_election.cc:343] T 25463905e343470d941f89a53efbc2d1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.278825 28617 leader_election.cc:304] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:44.292515 28346 leader_election.cc:304] T 25463905e343470d941f89a53efbc2d1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:44.293493 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c9ceb3af08fa4618b81cf6f89d7bd0df (table default.Table_21242eba3cb64c7c91082e296439cfe2 [id=10a4f5ef579746b68d8dd37d83f14c3e]). Expected version 1 got 0
I20260618 13:59:44.293711 28797 tablet_bootstrap.cc:492] T d3ed5c1137434fcc80ed0ca56a323be7 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:44.293840 28797 ts_tablet_manager.cc:1403] T d3ed5c1137434fcc80ed0ca56a323be7 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.082s user 0.016s sys 0.027s
I20260618 13:59:44.294567 30864 raft_consensus.cc:2749] T 25463905e343470d941f89a53efbc2d1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:44.294842 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:44.295027 30856 raft_consensus.cc:2804] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:44.295053 29566 tablet_replica.cc:333] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:44.295150 30856 raft_consensus.cc:697] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:44.295387 30856 consensus_queue.cc:237] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.297662 28484 leader_election.cc:304] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:44.297901 30552 raft_consensus.cc:2749] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:44.299124 28797 raft_consensus.cc:359] T d3ed5c1137434fcc80ed0ca56a323be7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.299275 28797 raft_consensus.cc:385] T d3ed5c1137434fcc80ed0ca56a323be7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.299196 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9ceb3af08fa4618b81cf6f89d7bd0df" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:44.299320 28797 raft_consensus.cc:740] T d3ed5c1137434fcc80ed0ca56a323be7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.299489 28797 consensus_queue.cc:260] T d3ed5c1137434fcc80ed0ca56a323be7 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.299911 28797 ts_tablet_manager.cc:1434] T d3ed5c1137434fcc80ed0ca56a323be7 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260618 13:59:44.300145 28797 tablet_bootstrap.cc:492] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:44.303126 28618 leader_election.cc:343] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:44.303644 28618 leader_election.cc:343] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:44.303894 28618 consensus_peers.cc:597] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c5d046e0421142cd84c5b12d6a434f53. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:44.304006 28618 leader_election.cc:343] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:44.305598 28346 consensus_peers.cc:597] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:44.307713 30552 raft_consensus.cc:493] T 8af2bec820b842b4bf6b57482007bd6a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.307830 30552 raft_consensus.cc:515] T 8af2bec820b842b4bf6b57482007bd6a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.308399 30552 leader_election.cc:290] T 8af2bec820b842b4bf6b57482007bd6a P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.309444 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af2bec820b842b4bf6b57482007bd6a" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:44.310817 30847 consensus_queue.cc:1048] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:44.312052 28797 tablet_bootstrap.cc:654] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.312466 28617 leader_election.cc:304] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:44.313158 30847 raft_consensus.cc:2804] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:44.313237 30847 raft_consensus.cc:493] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:44.313288 30847 raft_consensus.cc:3060] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.315577 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_c61ea74d5c6d4a718da2ab92f4a52397 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 2 got 0
I20260618 13:59:44.316054 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:44.316388 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:44.317222 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet e196dd3c7fe14524a50152643ddbc2ad (table default.Table_2dfebd62a146446f999f1a7a1bd1af79 [id=02ffed3d852845d6845fb5176f036fce]). Expected version 1 got 0
I20260618 13:59:44.317487 30847 raft_consensus.cc:515] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.317715 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:44.318049 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:44.318229 30847 leader_election.cc:290] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.295374 29566 raft_consensus.cc:2243] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:44.318769 29566 raft_consensus.cc:2272] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:44.319396 29566 ts_tablet_manager.cc:1916] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:44.320374 30843 tablet.cc:1724] T 8af2bec820b842b4bf6b57482007bd6a P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:44.321300 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038ea0e9c0b04db8b7ec56b065e3cc51" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:44.321556 28407 raft_consensus.cc:2468] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:44.329236 28797 tablet_bootstrap.cc:492] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:44.329412 28797 ts_tablet_manager.cc:1403] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.003s
I20260618 13:59:44.330323 28797 raft_consensus.cc:359] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.330611 28797 raft_consensus.cc:385] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.330689 28797 raft_consensus.cc:740] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.330941 28797 consensus_queue.cc:260] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.331588 28797 ts_tablet_manager.cc:1434] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:44.331931 28797 tablet_bootstrap.cc:492] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:44.334127 28797 tablet_bootstrap.cc:654] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.339818 28798 tablet_bootstrap.cc:492] T f695a7e6a925470787a708c365e6b86b P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:44.339972 28798 ts_tablet_manager.cc:1403] T f695a7e6a925470787a708c365e6b86b P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.066s user 0.003s sys 0.001s
I20260618 13:59:44.340736 28798 raft_consensus.cc:359] T f695a7e6a925470787a708c365e6b86b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:44.340920 28798 raft_consensus.cc:385] T f695a7e6a925470787a708c365e6b86b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.340982 28798 raft_consensus.cc:740] T f695a7e6a925470787a708c365e6b86b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.341215 28798 consensus_queue.cc:260] T f695a7e6a925470787a708c365e6b86b P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:44.345014 28798 ts_tablet_manager.cc:1434] T f695a7e6a925470787a708c365e6b86b P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260618 13:59:44.318280 29026 catalog_manager.cc:5696] T 5bbb71449d434d54b007d6f0da0796d7 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:44.347339 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:44.360023 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9ceb3af08fa4618b81cf6f89d7bd0df" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:44.360272 28407 raft_consensus.cc:3060] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.366488 28798 tablet_bootstrap.cc:492] T d0bcff62290c43c9be7cc496fbff6142 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:44.367362 28617 leader_election.cc:304] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:44.368191 28798 tablet_bootstrap.cc:654] T d0bcff62290c43c9be7cc496fbff6142 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.368602 30856 raft_consensus.cc:2804] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:44.368732 30856 raft_consensus.cc:493] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:44.368925 30856 raft_consensus.cc:3060] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.369071 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:44.369566 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:44.369853 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:44.370276 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_01e1e5741bb54f78996857f6b79fc5d7 [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 2 got 0
I20260618 13:59:44.371095 29026 catalog_manager.cc:5696] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:44.371552 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 10e9a417eecf4d33ac3bd9bd6325f447 (table default.Table_21426dbf385d47d2af5fc25e287ed0ed [id=0de5461053eb4517b15d6b91875fc04d]). Expected version 1 got 0
I20260618 13:59:44.371779 30856 raft_consensus.cc:515] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.372056 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 00b602a473a94c05a617fb9478300c74 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:44.372332 30856 leader_election.cc:290] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.373517 30847 consensus_queue.cc:1048] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:44.375741 28407 raft_consensus.cc:2468] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
W20260618 13:59:44.377737 28618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 73161.9us
W20260618 13:59:44.382740 28618 leader_election.cc:343] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.382854 28618 leader_election.cc:304] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:44.383481 30847 raft_consensus.cc:2749] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260618 13:59:44.384339 28618 leader_election.cc:343] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.384409 30853 consensus_queue.cc:1048] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260618 13:59:44.385466 28618 consensus_peers.cc:597] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:44.385814 28484 leader_election.cc:304] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:44.384665 28617 leader_election.cc:304] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:44.386161 30853 raft_consensus.cc:2804] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:44.386247 30552 raft_consensus.cc:2804] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:44.386314 30552 raft_consensus.cc:493] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:44.386339 30853 raft_consensus.cc:697] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:44.386379 30552 raft_consensus.cc:3060] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.386649 30853 consensus_queue.cc:237] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.389014 30552 raft_consensus.cc:515] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.389648 30552 leader_election.cc:290] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.390745 30847 raft_consensus.cc:493] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.390867 30847 raft_consensus.cc:515] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.391372 30847 leader_election.cc:290] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.391590 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038ea0e9c0b04db8b7ec56b065e3cc51" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:44.392163 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038ea0e9c0b04db8b7ec56b065e3cc51" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:44.392735 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9ceb3af08fa4618b81cf6f89d7bd0df" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:44.393532 30152 raft_consensus.cc:493] T 66ada05b01b2427da182998a557529ca P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.393656 30152 raft_consensus.cc:515] T 66ada05b01b2427da182998a557529ca P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.394218 30152 leader_election.cc:290] T 66ada05b01b2427da182998a557529ca P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:44.394214 30719 raft_consensus.cc:493] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.394346 30719 raft_consensus.cc:515] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.394806 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ada05b01b2427da182998a557529ca" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:44.394857 30719 leader_election.cc:290] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.395565 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ada05b01b2427da182998a557529ca" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:44.395725 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8af2bec820b842b4bf6b57482007bd6a" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:44.395746 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01c8463055e4526b3c75190066f30b6" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:44.396469 28539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6216474b74a14af6abf977e312b77" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:44.396555 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e196dd3c7fe14524a50152643ddbc2ad" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:44.396896 28618 consensus_peers.cc:597] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:44.397055 28618 leader_election.cc:343] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:44.397325 28618 leader_election.cc:343] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:44.397483 28346 leader_election.cc:343] T 66ada05b01b2427da182998a557529ca P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:44.397899 28346 leader_election.cc:343] T 66ada05b01b2427da182998a557529ca P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.398003 28346 leader_election.cc:304] T 66ada05b01b2427da182998a557529ca P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:44.398106 28618 leader_election.cc:343] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.398620 30262 raft_consensus.cc:493] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.398758 30262 raft_consensus.cc:515] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.399273 30262 leader_election.cc:290] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:44.399310 28618 leader_election.cc:343] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.399425 30262 raft_consensus.cc:2749] T 66ada05b01b2427da182998a557529ca P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:44.399459 28618 leader_election.cc:343] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.400315 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e6f3f8431cd49b5bf1306bc7609761d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:44.400586 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e6f3f8431cd49b5bf1306bc7609761d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:44.400835 28680 raft_consensus.cc:2468] T 0e6f3f8431cd49b5bf1306bc7609761d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:44.401108 28484 leader_election.cc:304] T 8af2bec820b842b4bf6b57482007bd6a P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:44.401468 28484 leader_election.cc:343] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: Tablet not RUNNING: STOPPED
I20260618 13:59:44.401929 30552 raft_consensus.cc:2749] T 8af2bec820b842b4bf6b57482007bd6a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:44.402344 28346 leader_election.cc:304] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:44.402575 30152 raft_consensus.cc:2804] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:44.402652 30152 raft_consensus.cc:493] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:44.402719 30152 raft_consensus.cc:3060] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.405208 30152 raft_consensus.cc:515] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.405764 30152 leader_election.cc:290] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:44.406124 28346 leader_election.cc:343] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.383000 29566 ts_tablet_manager.cc:1929] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:44.406602 29566 log.cc:1199] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/c01c8463055e4526b3c75190066f30b6
I20260618 13:59:44.406955 29566 ts_tablet_manager.cc:1950] T c01c8463055e4526b3c75190066f30b6 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
W20260618 13:59:44.406904 28483 leader_election.cc:343] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: c01c8463055e4526b3c75190066f30b6
I20260618 13:59:44.407037 28483 leader_election.cc:304] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:44.407307 30900 raft_consensus.cc:2749] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260618 13:59:44.407685 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 226) took 6908 ms. Trace:
I20260618 13:59:44.407765 29566 rpcz_store.cc:276] 0618 13:59:37.498823 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:37.498895 (+ 72us) service_pool.cc:224] Handling call
0618 13:59:44.286287 (+6787392us) ts_tablet_manager.cc:1151] Deleting tablet c01c8463055e4526b3c75190066f30b6
0618 13:59:44.295012 (+ 8725us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.347616 (+ 52604us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:44.382800 (+ 35184us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:44.407666 (+ 24866us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6787193}
I20260618 13:59:44.407912 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e6f3f8431cd49b5bf1306bc7609761d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:44.408044 28680 raft_consensus.cc:3060] T 0e6f3f8431cd49b5bf1306bc7609761d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.408082 29566 tablet_replica.cc:333] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:44.408274 29566 raft_consensus.cc:2243] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:44.408396 29566 raft_consensus.cc:2272] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:44.408881 29566 ts_tablet_manager.cc:1916] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:44.409587 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet c01c8463055e4526b3c75190066f30b6 (table default.Table_7e5d58ad3e8043919b66e8890819535c [id=96aff34b68c1494f9228a11eb38de4c7]) successfully deleted
I20260618 13:59:44.410200 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e6f3f8431cd49b5bf1306bc7609761d" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:44.410877 28346 leader_election.cc:343] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.411242 28797 tablet_bootstrap.cc:492] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:44.411362 28797 ts_tablet_manager.cc:1403] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.080s user 0.013s sys 0.037s
I20260618 13:59:44.412156 28797 raft_consensus.cc:359] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.412334 28797 raft_consensus.cc:385] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.412405 28797 raft_consensus.cc:740] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.412665 28797 consensus_queue.cc:260] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.413358 28797 ts_tablet_manager.cc:1434] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:44.413619 28797 tablet_bootstrap.cc:492] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:44.415236 28797 tablet_bootstrap.cc:654] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.414335 29026 catalog_manager.cc:5696] T 9d8f124ba79b46e098303b7b2705844d P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:44.419636 28680 raft_consensus.cc:2468] T 0e6f3f8431cd49b5bf1306bc7609761d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:44.422052 28346 leader_election.cc:304] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:44.422595 30152 raft_consensus.cc:2804] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:44.422856 30152 raft_consensus.cc:697] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:44.423216 30152 consensus_queue.cc:237] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.427242 28798 tablet_bootstrap.cc:492] T d0bcff62290c43c9be7cc496fbff6142 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:44.427438 28798 ts_tablet_manager.cc:1403] T d0bcff62290c43c9be7cc496fbff6142 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.082s user 0.002s sys 0.002s
I20260618 13:59:44.428129 28798 raft_consensus.cc:359] T d0bcff62290c43c9be7cc496fbff6142 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:44.428359 28798 raft_consensus.cc:385] T d0bcff62290c43c9be7cc496fbff6142 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.428457 28798 raft_consensus.cc:740] T d0bcff62290c43c9be7cc496fbff6142 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.428747 28798 consensus_queue.cc:260] T d0bcff62290c43c9be7cc496fbff6142 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:44.429808 28798 ts_tablet_manager.cc:1434] T d0bcff62290c43c9be7cc496fbff6142 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:44.430125 28798 tablet_bootstrap.cc:492] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:44.432060 28798 tablet_bootstrap.cc:654] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.432521 30262 raft_consensus.cc:493] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.432662 30262 raft_consensus.cc:515] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.433193 30262 leader_election.cc:290] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:44.433733 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:44.434243 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 8af2bec820b842b4bf6b57482007bd6a (table default.Table_84be6926f4b64876b343577b85db3243 [id=af1859c644b44c1d831314d27a494f94]). Expected version 1 got 0
I20260618 13:59:44.434671 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:44.434842 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e196dd3c7fe14524a50152643ddbc2ad" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:44.434926 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:44.434999 28402 raft_consensus.cc:2468] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:44.435214 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6216474b74a14af6abf977e312b77" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:44.435381 28404 raft_consensus.cc:2468] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:44.435283 29026 catalog_manager.cc:5696] T c871e7c8f843486da7ba40145c0fed19 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:44.435672 30892 tablet.cc:1724] T 8af2bec820b842b4bf6b57482007bd6a P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:44.436520 29026 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:44.438182 30577 raft_consensus.cc:493] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.438342 30577 raft_consensus.cc:515] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.439155 30577 leader_election.cc:290] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.440330 28401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2a6fabad00642528913e735929e295e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:44.440482 28401 raft_consensus.cc:2468] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:44.442947 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2a6fabad00642528913e735929e295e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:44.443979 28618 leader_election.cc:343] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:44.444236 28618 consensus_peers.cc:597] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:44.444515 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "038ea0e9c0b04db8b7ec56b065e3cc51" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:44.444697 28407 raft_consensus.cc:3060] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.447618 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ed5c1137434fcc80ed0ca56a323be7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:44.449028 28618 consensus_peers.cc:597] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:44.449560 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ed5c1137434fcc80ed0ca56a323be7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:44.449757 28680 raft_consensus.cc:2468] T d3ed5c1137434fcc80ed0ca56a323be7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:44.450738 28346 leader_election.cc:304] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:44.451001 30262 raft_consensus.cc:2804] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:44.451063 30262 raft_consensus.cc:493] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:44.451116 30262 raft_consensus.cc:3060] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.452634 28617 leader_election.cc:304] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:44.452979 30577 raft_consensus.cc:2804] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:44.453044 30577 raft_consensus.cc:493] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:44.453106 30577 raft_consensus.cc:3060] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.454015 30262 raft_consensus.cc:515] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.454905 30262 leader_election.cc:290] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:44.456101 30577 raft_consensus.cc:515] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.456631 30577 leader_election.cc:290] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.457409 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e196dd3c7fe14524a50152643ddbc2ad" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:44.457928 28618 leader_election.cc:343] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:44.458396 28346 leader_election.cc:343] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.458709 28617 leader_election.cc:304] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:44.459456 30890 raft_consensus.cc:2804] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:44.459647 30890 raft_consensus.cc:493] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:44.459795 30890 raft_consensus.cc:3060] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.460327 28617 leader_election.cc:304] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:44.460978 30719 raft_consensus.cc:2804] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:44.461052 30719 raft_consensus.cc:493] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:44.461141 30719 raft_consensus.cc:3060] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.462632 29566 ts_tablet_manager.cc:1929] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:44.462718 29566 log.cc:1199] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/6c4a31a4f202473d97b17b5b5dfdb615
I20260618 13:59:44.462999 29566 ts_tablet_manager.cc:1950] T 6c4a31a4f202473d97b17b5b5dfdb615 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:44.463594 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 227) took 6942 ms. Trace:
I20260618 13:59:44.463626 30719 raft_consensus.cc:515] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.463661 29566 rpcz_store.cc:276] 0618 13:59:37.520909 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:37.520964 (+ 55us) service_pool.cc:224] Handling call
0618 13:59:44.408040 (+6887076us) ts_tablet_manager.cc:1151] Deleting tablet 6c4a31a4f202473d97b17b5b5dfdb615
0618 13:59:44.408049 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.440666 (+ 32617us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:44.462581 (+ 21915us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:44.463573 (+ 992us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6886897}
I20260618 13:59:44.463970 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 231) took 4432 ms. Trace:
I20260618 13:59:44.464025 29566 rpcz_store.cc:276] 0618 13:59:40.031911 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.032021 (+ 110us) service_pool.cc:224] Handling call
0618 13:59:44.463896 (+4431875us) ts_tablet_manager.cc:1151] Deleting tablet 2f5a334d18ba4c06bb358fa86d8af80d
0618 13:59:44.463904 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.463959 (+ 55us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4431637}
I20260618 13:59:44.464182 30719 leader_election.cc:290] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.464150 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 6c4a31a4f202473d97b17b5b5dfdb615 (table default.Table_152a483501b54a67bbdc0cae4959d452 [id=daf2b982b88e4021b327a0e52cf5316c]) successfully deleted
I20260618 13:59:44.464262 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 232) took 4432 ms. Trace:
I20260618 13:59:44.464316 29566 rpcz_store.cc:276] 0618 13:59:40.032214 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.032274 (+ 60us) service_pool.cc:224] Handling call
0618 13:59:44.464199 (+4431925us) ts_tablet_manager.cc:1151] Deleting tablet 516421cd843b44769efa2b555697c065
0618 13:59:44.464206 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.464250 (+ 44us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4431749}
I20260618 13:59:44.464521 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 234) took 4431 ms. Trace:
I20260618 13:59:44.464589 29566 rpcz_store.cc:276] 0618 13:59:40.033531 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.033577 (+ 46us) service_pool.cc:224] Handling call
0618 13:59:44.464470 (+4430893us) ts_tablet_manager.cc:1151] Deleting tablet 7d4144b1b5ea4e968672616bc22bead8
0618 13:59:44.464476 (+ 6us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.464512 (+ 36us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4430722}
W20260618 13:59:44.465051 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 2f5a334d18ba4c06bb358fa86d8af80d because the tablet was not found. No further retry: Not found: Tablet not found: 2f5a334d18ba4c06bb358fa86d8af80d
I20260618 13:59:44.465081 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2a6fabad00642528913e735929e295e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:44.465286 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 516421cd843b44769efa2b555697c065 because the tablet was not found. No further retry: Not found: Tablet not found: 516421cd843b44769efa2b555697c065
W20260618 13:59:44.465482 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 7d4144b1b5ea4e968672616bc22bead8 because the tablet was not found. No further retry: Not found: Tablet not found: 7d4144b1b5ea4e968672616bc22bead8
W20260618 13:59:44.465615 28618 leader_election.cc:343] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.465684 28798 tablet_bootstrap.cc:492] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:44.465821 28798 ts_tablet_manager.cc:1403] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.000s
I20260618 13:59:44.466480 28798 raft_consensus.cc:359] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:44.466632 28798 raft_consensus.cc:385] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.466686 28798 raft_consensus.cc:740] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.466895 28798 consensus_queue.cc:260] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:44.467511 28798 ts_tablet_manager.cc:1434] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:44.467763 28798 tablet_bootstrap.cc:492] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:44.469051 28798 tablet_bootstrap.cc:654] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.470625 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ed5c1137434fcc80ed0ca56a323be7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:44.471253 28407 raft_consensus.cc:2468] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:44.471475 30847 raft_consensus.cc:493] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.471585 30847 raft_consensus.cc:515] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.471858 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3ed5c1137434fcc80ed0ca56a323be7" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:44.472059 28680 raft_consensus.cc:3060] T d3ed5c1137434fcc80ed0ca56a323be7 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.472180 30847 leader_election.cc:290] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.475502 28680 raft_consensus.cc:2468] T d3ed5c1137434fcc80ed0ca56a323be7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
W20260618 13:59:44.475898 28346 leader_election.cc:343] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.478847 28798 tablet_bootstrap.cc:492] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:44.478958 28798 ts_tablet_manager.cc:1403] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260618 13:59:44.479071 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c50ab60a26647558da509236aac1e8b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:44.479511 28798 raft_consensus.cc:359] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:44.479640 28798 raft_consensus.cc:385] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.479681 28798 raft_consensus.cc:740] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.479882 28798 consensus_queue.cc:260] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:44.480427 28798 ts_tablet_manager.cc:1434] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260618 13:59:44.480641 28798 tablet_bootstrap.cc:492] T 9d8f124ba79b46e098303b7b2705844d P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:44.482268 28798 tablet_bootstrap.cc:654] T 9d8f124ba79b46e098303b7b2705844d P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.487723 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2a6fabad00642528913e735929e295e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:44.487898 28404 raft_consensus.cc:3060] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.490087 28798 tablet_bootstrap.cc:492] T 9d8f124ba79b46e098303b7b2705844d P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:44.490224 28798 ts_tablet_manager.cc:1403] T 9d8f124ba79b46e098303b7b2705844d P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260618 13:59:44.491007 28798 raft_consensus.cc:359] T 9d8f124ba79b46e098303b7b2705844d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.491197 28798 raft_consensus.cc:385] T 9d8f124ba79b46e098303b7b2705844d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.491264 28798 raft_consensus.cc:740] T 9d8f124ba79b46e098303b7b2705844d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.491528 28798 consensus_queue.cc:260] T 9d8f124ba79b46e098303b7b2705844d P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.491874 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e196dd3c7fe14524a50152643ddbc2ad" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:44.492080 28408 raft_consensus.cc:3060] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.492373 28798 ts_tablet_manager.cc:1434] T 9d8f124ba79b46e098303b7b2705844d P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:44.492678 28798 tablet_bootstrap.cc:492] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:44.494328 28798 tablet_bootstrap.cc:654] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.495930 28408 raft_consensus.cc:2468] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:44.497731 28404 raft_consensus.cc:2468] T b2a6fabad00642528913e735929e295e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:44.500732 30262 raft_consensus.cc:493] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.500924 30262 raft_consensus.cc:515] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.501498 30262 leader_election.cc:290] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:44.496658 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c50ab60a26647558da509236aac1e8b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:44.502323 28408 raft_consensus.cc:2468] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:44.503361 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba18e879631947f88bd266681746ec82" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:44.504057 30622 raft_consensus.cc:493] T 6827859dd7be4464a9d98ce502df8539 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.504154 30622 raft_consensus.cc:515] T 6827859dd7be4464a9d98ce502df8539 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.504662 30622 leader_election.cc:290] T 6827859dd7be4464a9d98ce502df8539 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:44.503903 30864 consensus_queue.cc:1048] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260618 13:59:44.506556 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6827859dd7be4464a9d98ce502df8539" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:44.506568 28681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba18e879631947f88bd266681746ec82" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:44.478502 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 235) took 4444 ms. Trace:
I20260618 13:59:44.477085 30890 raft_consensus.cc:515] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.488997 28617 leader_election.cc:304] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:44.506986 29566 rpcz_store.cc:276] 0618 13:59:40.034033 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.034085 (+ 52us) service_pool.cc:224] Handling call
0618 13:59:44.464734 (+4430649us) ts_tablet_manager.cc:1151] Deleting tablet 4c2b924a096b479cad31c29bf7c99172
0618 13:59:44.464740 (+ 6us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.478486 (+ 13746us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4430488}
I20260618 13:59:44.508586 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 236) took 4474 ms. Trace:
W20260618 13:59:44.508595 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 4c2b924a096b479cad31c29bf7c99172 because the tablet was not found. No further retry: Not found: Tablet not found: 4c2b924a096b479cad31c29bf7c99172
I20260618 13:59:44.508710 29566 rpcz_store.cc:276] 0618 13:59:40.034557 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.034608 (+ 51us) service_pool.cc:224] Handling call
0618 13:59:44.508493 (+4473885us) ts_tablet_manager.cc:1151] Deleting tablet da4d4f03911f4ac099ecc2fa1d8af494
0618 13:59:44.508503 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.508572 (+ 69us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4473724}
I20260618 13:59:44.509107 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 237) took 4474 ms. Trace:
W20260618 13:59:44.509095 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet da4d4f03911f4ac099ecc2fa1d8af494 because the tablet was not found. No further retry: Not found: Tablet not found: da4d4f03911f4ac099ecc2fa1d8af494
I20260618 13:59:44.509176 29566 rpcz_store.cc:276] 0618 13:59:40.035062 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.035113 (+ 51us) service_pool.cc:224] Handling call
0618 13:59:44.509037 (+4473924us) ts_tablet_manager.cc:1151] Deleting tablet aa8cf927bed9437086efd8e0f21ba26a
0618 13:59:44.509046 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.509094 (+ 48us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4473756}
W20260618 13:59:44.509662 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet aa8cf927bed9437086efd8e0f21ba26a because the tablet was not found. No further retry: Not found: Tablet not found: aa8cf927bed9437086efd8e0f21ba26a
I20260618 13:59:44.509742 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 238) took 4474 ms. Trace:
I20260618 13:59:44.509799 29566 rpcz_store.cc:276] 0618 13:59:40.035742 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.035795 (+ 53us) service_pool.cc:224] Handling call
0618 13:59:44.509667 (+4473872us) ts_tablet_manager.cc:1151] Deleting tablet c557afc3abfa4ef188bf248e89f49d87
0618 13:59:44.509677 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.509727 (+ 50us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4473694}
W20260618 13:59:44.510141 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet c557afc3abfa4ef188bf248e89f49d87 because the tablet was not found. No further retry: Not found: Tablet not found: c557afc3abfa4ef188bf248e89f49d87
I20260618 13:59:44.510227 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 239) took 4474 ms. Trace:
I20260618 13:59:44.510290 29566 rpcz_store.cc:276] 0618 13:59:40.036249 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.036296 (+ 47us) service_pool.cc:224] Handling call
0618 13:59:44.510157 (+4473861us) ts_tablet_manager.cc:1151] Deleting tablet c7f42997416841a7b2c1ad347d2e9282
0618 13:59:44.510167 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.510213 (+ 46us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4473699}
I20260618 13:59:44.510643 29566 tablet_replica.cc:333] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
W20260618 13:59:44.510784 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet c7f42997416841a7b2c1ad347d2e9282 because the tablet was not found. No further retry: Not found: Tablet not found: c7f42997416841a7b2c1ad347d2e9282
I20260618 13:59:44.510848 29566 raft_consensus.cc:2243] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:44.510983 29566 raft_consensus.cc:2272] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:44.511447 29566 ts_tablet_manager.cc:1916] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:44.482291 28618 leader_election.cc:343] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:44.515767 28618 consensus_peers.cc:597] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:44.507630 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6827859dd7be4464a9d98ce502df8539" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:44.516109 28537 raft_consensus.cc:3060] T f695a7e6a925470787a708c365e6b86b P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.516539 30853 raft_consensus.cc:2804] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:44.516660 30853 raft_consensus.cc:697] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:44.516886 30853 consensus_queue.cc:237] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.517879 29566 ts_tablet_manager.cc:1929] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:44.518982 29566 log.cc:1199] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/d4667fe48f0f4eff9c6c8baabdab58d4
I20260618 13:59:44.519364 28617 leader_election.cc:304] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:44.519448 29566 ts_tablet_manager.cc:1950] T d4667fe48f0f4eff9c6c8baabdab58d4 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:44.519618 30577 raft_consensus.cc:2804] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:44.519759 30577 raft_consensus.cc:697] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:44.519982 30577 consensus_queue.cc:237] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.520534 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 240) took 4483 ms. Trace:
I20260618 13:59:44.520651 29566 rpcz_store.cc:276] 0618 13:59:40.036719 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.036768 (+ 49us) service_pool.cc:224] Handling call
0618 13:59:44.510597 (+4473829us) ts_tablet_manager.cc:1151] Deleting tablet d4667fe48f0f4eff9c6c8baabdab58d4
0618 13:59:44.510607 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.513547 (+ 2940us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:44.517836 (+ 4289us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:44.520519 (+ 2683us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4473663}
I20260618 13:59:44.521167 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet d4667fe48f0f4eff9c6c8baabdab58d4 (table default.Table_0a85d79af65f48a3803bfd1248593a11 [id=2f4d62be772a421ab777fd23be58b719]) successfully deleted
I20260618 13:59:44.521648 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 241) took 4484 ms. Trace:
I20260618 13:59:44.522634 29566 rpcz_store.cc:276] 0618 13:59:40.037219 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.037268 (+ 49us) service_pool.cc:224] Handling call
0618 13:59:44.521553 (+4484285us) ts_tablet_manager.cc:1151] Deleting tablet b16f7d3c7837498e996bba1818110642
0618 13:59:44.521565 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.521630 (+ 65us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4484124}
W20260618 13:59:44.524030 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet b16f7d3c7837498e996bba1818110642 because the tablet was not found. No further retry: Not found: Tablet not found: b16f7d3c7837498e996bba1818110642
I20260618 13:59:44.523419 30847 consensus_queue.cc:1048] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:44.524833 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 242) took 4487 ms. Trace:
I20260618 13:59:44.525166 29566 rpcz_store.cc:276] 0618 13:59:40.037695 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.037744 (+ 49us) service_pool.cc:224] Handling call
0618 13:59:44.524756 (+4487012us) ts_tablet_manager.cc:1151] Deleting tablet bc0adedecee4405e86b0a37b9e62e352
0618 13:59:44.524765 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.524817 (+ 52us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4486852}
I20260618 13:59:44.525694 28617 leader_election.cc:304] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:44.461138 28797 tablet_bootstrap.cc:492] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
W20260618 13:59:44.525869 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet bc0adedecee4405e86b0a37b9e62e352 because the tablet was not found. No further retry: Not found: Tablet not found: bc0adedecee4405e86b0a37b9e62e352
I20260618 13:59:44.526014 28797 ts_tablet_manager.cc:1403] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.112s user 0.000s sys 0.008s
I20260618 13:59:44.526270 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 243) took 4488 ms. Trace:
I20260618 13:59:44.528648 29566 rpcz_store.cc:276] 0618 13:59:40.038179 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.038226 (+ 47us) service_pool.cc:224] Handling call
0618 13:59:44.526182 (+4487956us) ts_tablet_manager.cc:1151] Deleting tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9
0618 13:59:44.526190 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.526249 (+ 59us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4487785}
I20260618 13:59:44.529047 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 244) took 4490 ms. Trace:
I20260618 13:59:44.529160 29566 rpcz_store.cc:276] 0618 13:59:40.038951 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.039003 (+ 52us) service_pool.cc:224] Handling call
0618 13:59:44.528971 (+4489968us) ts_tablet_manager.cc:1151] Deleting tablet 0fae928833e44d2ba860d6184d4c0142
0618 13:59:44.528980 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.529031 (+ 51us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4489807}
I20260618 13:59:44.529717 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6216474b74a14af6abf977e312b77" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:44.529286 30847 raft_consensus.cc:2804] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:44.530300 28346 leader_election.cc:304] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:44.529505 30890 leader_election.cc:290] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:44.530242 28618 leader_election.cc:343] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.531445 30847 raft_consensus.cc:697] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:44.531778 30847 consensus_queue.cc:237] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.533020 30854 raft_consensus.cc:493] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.533154 30854 raft_consensus.cc:515] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.530670 29566 tablet_replica.cc:333] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:44.533865 30854 leader_election.cc:290] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:44.534238 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 because the tablet was not found. No further retry: Not found: Tablet not found: 75b9c0754beb4e0e8ed3a7c1e9f368a9
I20260618 13:59:44.534533 29566 raft_consensus.cc:2243] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
W20260618 13:59:44.534530 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 0fae928833e44d2ba860d6184d4c0142 because the tablet was not found. No further retry: Not found: Tablet not found: 0fae928833e44d2ba860d6184d4c0142
I20260618 13:59:44.535571 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5115a31a38db48c1a6474917aff97c8e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:44.536191 28618 leader_election.cc:343] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.536684 29566 raft_consensus.cc:2272] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:44.537984 30152 raft_consensus.cc:2804] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:44.538113 30152 raft_consensus.cc:697] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:44.538321 30152 consensus_queue.cc:237] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.537308 28617 leader_election.cc:304] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:44.543394 30856 raft_consensus.cc:2804] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:44.543543 30856 raft_consensus.cc:493] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:44.543677 30856 raft_consensus.cc:3060] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.546747 30856 raft_consensus.cc:515] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:44.544492 28346 leader_election.cc:343] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:44.547401 28346 leader_election.cc:343] T 6827859dd7be4464a9d98ce502df8539 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:44.547602 28346 leader_election.cc:343] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.547688 28346 leader_election.cc:304] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:44.548025 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c50ab60a26647558da509236aac1e8b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:44.548593 30856 leader_election.cc:290] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:44.548560 28618 leader_election.cc:343] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.543681 29566 ts_tablet_manager.cc:1916] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:44.552647 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6216474b74a14af6abf977e312b77" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:44.552804 28407 raft_consensus.cc:3060] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.556324 28407 raft_consensus.cc:2468] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:44.537698 28797 raft_consensus.cc:359] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.561278 28797 raft_consensus.cc:385] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.561534 28797 raft_consensus.cc:740] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.562090 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5115a31a38db48c1a6474917aff97c8e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:44.562165 30864 raft_consensus.cc:2749] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:44.562458 28407 raft_consensus.cc:2468] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:44.567890 28797 consensus_queue.cc:260] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:44.568502 28346 leader_election.cc:343] T 6827859dd7be4464a9d98ce502df8539 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.568563 28797 ts_tablet_manager.cc:1434] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.042s user 0.001s sys 0.001s
I20260618 13:59:44.568629 28798 tablet_bootstrap.cc:492] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:44.568624 28346 leader_election.cc:304] T 6827859dd7be4464a9d98ce502df8539 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:44.568748 28798 ts_tablet_manager.cc:1403] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.076s user 0.017s sys 0.055s
W20260618 13:59:44.569020 28346 consensus_peers.cc:597] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:44.569398 28797 tablet_bootstrap.cc:492] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:44.569753 28798 raft_consensus.cc:359] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.569916 28798 raft_consensus.cc:385] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.569965 28798 raft_consensus.cc:740] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.569972 30152 raft_consensus.cc:2749] T 6827859dd7be4464a9d98ce502df8539 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:44.570148 28798 consensus_queue.cc:260] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.570806 28798 ts_tablet_manager.cc:1434] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:44.571096 28798 tablet_bootstrap.cc:492] T a3c74d3ca66f42ad83ef54d9459b99fd P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:44.572021 28797 tablet_bootstrap.cc:654] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.572706 28798 tablet_bootstrap.cc:654] T a3c74d3ca66f42ad83ef54d9459b99fd P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.572933 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c50ab60a26647558da509236aac1e8b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:44.573184 28407 raft_consensus.cc:3060] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.576141 28407 raft_consensus.cc:2468] T 7c50ab60a26647558da509236aac1e8b P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:44.542272 29017 catalog_manager.cc:5696] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:44.577723 28617 leader_election.cc:304] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:44.577752 29017 catalog_manager.cc:5696] T e3a3395aa90e426eb48a3920a8c45ba8 P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:44.578176 28617 leader_election.cc:304] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:44.578528 28617 leader_election.cc:304] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:44.578786 30572 raft_consensus.cc:493] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.578799 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:44.578929 30854 raft_consensus.cc:2804] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:44.578899 30572 raft_consensus.cc:515] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.579015 30854 raft_consensus.cc:697] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:44.579223 30854 consensus_queue.cc:237] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.579440 30572 leader_election.cc:290] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.579566 30572 raft_consensus.cc:2804] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:44.579619 30572 raft_consensus.cc:493] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:44.579619 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:44.579672 30572 raft_consensus.cc:3060] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.579922 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 8af2bec820b842b4bf6b57482007bd6a (table default.Table_84be6926f4b64876b343577b85db3243 [id=af1859c644b44c1d831314d27a494f94]). Expected version 1 got 0
I20260618 13:59:44.580647 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:44.581216 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:44.581653 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:44.582708 29303 raft_consensus.cc:2804] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:44.582858 29303 raft_consensus.cc:697] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:44.583135 29303 consensus_queue.cc:237] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.588979 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea86899d2c047d997cebe80551e83c9" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:44.590924 30572 raft_consensus.cc:515] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.591814 30572 leader_election.cc:290] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:44.592571 28618 leader_election.cc:343] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.593176 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5115a31a38db48c1a6474917aff97c8e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:44.594099 28618 leader_election.cc:343] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.602233 28548 raft_consensus.cc:3060] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.608048 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5115a31a38db48c1a6474917aff97c8e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:44.608306 28407 raft_consensus.cc:3060] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.608966 30622 raft_consensus.cc:493] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.609081 30622 raft_consensus.cc:515] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:44.609622 30622 leader_election.cc:290] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:44.610458 30900 raft_consensus.cc:493] T 9d8f124ba79b46e098303b7b2705844d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.582396 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:44.610591 30900 raft_consensus.cc:515] T 9d8f124ba79b46e098303b7b2705844d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.610901 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:44.611127 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet e196dd3c7fe14524a50152643ddbc2ad (table default.Table_2dfebd62a146446f999f1a7a1bd1af79 [id=02ffed3d852845d6845fb5176f036fce]). Expected version 1 got 0
I20260618 13:59:44.611260 30900 leader_election.cc:290] T 9d8f124ba79b46e098303b7b2705844d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.611493 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_01e1e5741bb54f78996857f6b79fc5d7 [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 2 got 0
I20260618 13:59:44.611901 28407 raft_consensus.cc:2468] T 5115a31a38db48c1a6474917aff97c8e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:44.612339 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:44.612995 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:44.614095 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 00b602a473a94c05a617fb9478300c74 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:44.615100 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:44.622532 30152 raft_consensus.cc:493] T 43853b8aba3444c3a21f6416f224e0d6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.622731 30152 raft_consensus.cc:515] T 43853b8aba3444c3a21f6416f224e0d6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.623478 30152 leader_election.cc:290] T 43853b8aba3444c3a21f6416f224e0d6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:44.624219 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8f124ba79b46e098303b7b2705844d" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:44.627041 28798 tablet_bootstrap.cc:492] T a3c74d3ca66f42ad83ef54d9459b99fd P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:44.627199 28798 ts_tablet_manager.cc:1403] T a3c74d3ca66f42ad83ef54d9459b99fd P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.056s user 0.017s sys 0.037s
I20260618 13:59:44.627823 28798 raft_consensus.cc:359] T a3c74d3ca66f42ad83ef54d9459b99fd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.628072 28798 raft_consensus.cc:385] T a3c74d3ca66f42ad83ef54d9459b99fd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.628127 28798 raft_consensus.cc:740] T a3c74d3ca66f42ad83ef54d9459b99fd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.628319 28798 consensus_queue.cc:260] T a3c74d3ca66f42ad83ef54d9459b99fd P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.628764 28798 ts_tablet_manager.cc:1434] T a3c74d3ca66f42ad83ef54d9459b99fd P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:44.629009 28798 tablet_bootstrap.cc:492] T c871e7c8f843486da7ba40145c0fed19 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:44.630702 28798 tablet_bootstrap.cc:654] T c871e7c8f843486da7ba40145c0fed19 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.634763 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:44.634934 30864 consensus_queue.cc:1048] T a662bb37fb7d4df49e61e524808b67a8 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:44.635696 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f9ec5f31f6c4d37b7c3ec585ed3211b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:44.636121 28617 leader_election.cc:343] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: fea86899d2c047d997cebe80551e83c9
I20260618 13:59:44.637890 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f9ec5f31f6c4d37b7c3ec585ed3211b" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:44.643479 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43853b8aba3444c3a21f6416f224e0d6" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:44.644086 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8f124ba79b46e098303b7b2705844d" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:44.644557 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c9ceb3af08fa4618b81cf6f89d7bd0df (table default.Table_21242eba3cb64c7c91082e296439cfe2 [id=10a4f5ef579746b68d8dd37d83f14c3e]). Expected version 1 got 0
I20260618 13:59:44.644891 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43853b8aba3444c3a21f6416f224e0d6" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:44.636291 28617 leader_election.cc:304] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:44.646473 28798 tablet_bootstrap.cc:492] T c871e7c8f843486da7ba40145c0fed19 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:44.646603 28798 ts_tablet_manager.cc:1403] T c871e7c8f843486da7ba40145c0fed19 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.003s
I20260618 13:59:44.646973 28617 leader_election.cc:304] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:44.646917 29017 catalog_manager.cc:5696] T ac9f580e35af4d4080d313b0283c0b7e P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:44.647526 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:44.647634 30572 raft_consensus.cc:2749] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:44.647855 28798 raft_consensus.cc:359] T c871e7c8f843486da7ba40145c0fed19 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.648013 28798 raft_consensus.cc:385] T c871e7c8f843486da7ba40145c0fed19 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.648074 28798 raft_consensus.cc:740] T c871e7c8f843486da7ba40145c0fed19 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.648277 28798 consensus_queue.cc:260] T c871e7c8f843486da7ba40145c0fed19 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.648998 28798 ts_tablet_manager.cc:1434] T c871e7c8f843486da7ba40145c0fed19 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:44.649259 28798 tablet_bootstrap.cc:492] T 3f5d4296952e48a48756e6f27140da87 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:44.651892 28798 tablet_bootstrap.cc:654] T 3f5d4296952e48a48756e6f27140da87 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.652029 30900 raft_consensus.cc:493] T c871e7c8f843486da7ba40145c0fed19 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.652127 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:44.652156 30900 raft_consensus.cc:515] T c871e7c8f843486da7ba40145c0fed19 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.652442 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:44.652704 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 10e9a417eecf4d33ac3bd9bd6325f447 (table default.Table_21426dbf385d47d2af5fc25e287ed0ed [id=0de5461053eb4517b15d6b91875fc04d]). Expected version 1 got 0
I20260618 13:59:44.652756 30900 leader_election.cc:290] T c871e7c8f843486da7ba40145c0fed19 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.653162 29017 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 438663f365e34fab87e34f434920cecf (table default.Table_c61ea74d5c6d4a718da2ab92f4a52397 [id=aec77306ae6d47ec91b286291c3ab664]). Expected version 2 got 0
I20260618 13:59:44.653476 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c871e7c8f843486da7ba40145c0fed19" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
W20260618 13:59:44.656355 28346 leader_election.cc:343] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:44.658244 28346 leader_election.cc:343] T 43853b8aba3444c3a21f6416f224e0d6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.659145 28484 leader_election.cc:304] T 9d8f124ba79b46e098303b7b2705844d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:44.659592 30900 raft_consensus.cc:2749] T 9d8f124ba79b46e098303b7b2705844d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:44.659968 28537 raft_consensus.cc:3060] T 8f7b66652f464893ad812d27e883a9a4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.666597 29011 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_16516271526f49bf92301a70916453c1 [id=0321ff97d1674a3891eae8cbba76387a] alter complete (version 1)
W20260618 13:59:44.668174 28346 leader_election.cc:343] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.668295 28346 leader_election.cc:304] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:44.670523 30864 raft_consensus.cc:2749] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:44.670794 30864 raft_consensus.cc:493] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.670941 30864 raft_consensus.cc:515] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.671586 30864 leader_election.cc:290] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:44.671854 28346 leader_election.cc:343] T 43853b8aba3444c3a21f6416f224e0d6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.671959 28346 leader_election.cc:304] T 43853b8aba3444c3a21f6416f224e0d6 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:44.676690 28596 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:44.678905 30864 raft_consensus.cc:2749] T 43853b8aba3444c3a21f6416f224e0d6 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:44.682529 29024 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_93fe53cac2f54d73b1e66cf1661ca928 [id=333c0fd3837b4d89a9197eb6a826c311] alter complete (version 3)
I20260618 13:59:44.685963 29011 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_c20f026cc38841858fb9df9a0a01f852 [id=a508a0cb72474a748aacdfd918e3782a] alter complete (version 1)
I20260618 13:59:44.686277 29007 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_6454f30d08d14030a0a405bcad4e06b3 [id=bf0b6154187f472a941e2aa04f635522] alter complete (version 1)
I20260618 13:59:44.694631 30262 raft_consensus.cc:493] T 134a43259d5b429fbe47d0555d9942f3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.694816 30262 raft_consensus.cc:515] T 134a43259d5b429fbe47d0555d9942f3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.695475 30262 leader_election.cc:290] T 134a43259d5b429fbe47d0555d9942f3 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:44.712533 28798 tablet_bootstrap.cc:492] T 3f5d4296952e48a48756e6f27140da87 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:44.712688 28798 ts_tablet_manager.cc:1403] T 3f5d4296952e48a48756e6f27140da87 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.063s user 0.013s sys 0.016s
I20260618 13:59:44.713353 28798 raft_consensus.cc:359] T 3f5d4296952e48a48756e6f27140da87 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.713536 28798 raft_consensus.cc:385] T 3f5d4296952e48a48756e6f27140da87 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.713588 28798 raft_consensus.cc:740] T 3f5d4296952e48a48756e6f27140da87 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.713778 28798 consensus_queue.cc:260] T 3f5d4296952e48a48756e6f27140da87 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.682494 29026 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_98d3bc8b4a7044d685c351a61e02f3e3 [id=5367efaf388c4ed78132d9752c341d5b] alter complete (version 1)
I20260618 13:59:44.716346 28798 ts_tablet_manager.cc:1434] T 3f5d4296952e48a48756e6f27140da87 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 13:59:44.717187 28798 tablet_bootstrap.cc:492] T 5bbb71449d434d54b007d6f0da0796d7 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:44.720232 28798 tablet_bootstrap.cc:654] T 5bbb71449d434d54b007d6f0da0796d7 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.723457 29023 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_68bcaa0c160d4b4eb47a8a0b6b04c6d9 [id=979317e20c204b109fb05c4128c0e2a3] alter complete (version 2)
I20260618 13:59:44.724403 29024 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_b7f097c1abe1410a800ce5aa25771740 [id=6c30b030a60b47e2be48dfe6e9cfdfdb] alter complete (version 1)
I20260618 13:59:44.734264 30847 raft_consensus.cc:493] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.736342 30572 raft_consensus.cc:2804] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:44.741140 28458 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:44.744346 28797 tablet_bootstrap.cc:492] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:44.745941 30847 raft_consensus.cc:515] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:44.747399 28618 consensus_peers.cc:597] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:44.747511 30572 raft_consensus.cc:697] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:44.747929 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c871e7c8f843486da7ba40145c0fed19" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:44.748638 28484 leader_election.cc:304] T c871e7c8f843486da7ba40145c0fed19 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:44.749023 30900 raft_consensus.cc:2749] T c871e7c8f843486da7ba40145c0fed19 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:44.758553 28797 ts_tablet_manager.cc:1403] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.175s user 0.013s sys 0.021s
I20260618 13:59:44.759323 28797 raft_consensus.cc:359] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.759506 28797 raft_consensus.cc:385] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.759583 28797 raft_consensus.cc:740] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.759774 28797 consensus_queue.cc:260] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.760221 28797 ts_tablet_manager.cc:1434] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:44.760511 28797 tablet_bootstrap.cc:492] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:44.763573 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b91f053bcc42bb829bb8180e232c8e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:44.766132 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b91f053bcc42bb829bb8180e232c8e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:44.766593 28408 raft_consensus.cc:2468] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:44.768867 28346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.3:35503, user_credentials={real_user=slave}} blocked reactor thread for 89444.8us
I20260618 13:59:44.770251 30572 consensus_queue.cc:237] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.770640 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89b26d1595642ff82381f8baddfb99a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:44.772094 29019 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_6911f12e0ef94ed8a9589f9294ccab4f [id=f914bcce8551443caa218e7a92967307] alter complete (version 1)
I20260618 13:59:44.775028 30847 leader_election.cc:290] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:44.775178 28618 leader_election.cc:343] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.781281 30900 raft_consensus.cc:493] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.781471 30900 raft_consensus.cc:515] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.782150 30900 leader_election.cc:290] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:44.782932 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fea86ca38a741238a59053151087f69" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:44.783277 28682 raft_consensus.cc:2468] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:44.784353 28484 leader_election.cc:304] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:44.787415 29566 ts_tablet_manager.cc:1929] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:44.787583 29566 log.cc:1199] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/f2b21dcf74fa4264947ca43cc0026e76
I20260618 13:59:44.788182 29566 ts_tablet_manager.cc:1950] T f2b21dcf74fa4264947ca43cc0026e76 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:44.790567 30900 raft_consensus.cc:2804] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:44.790673 30900 raft_consensus.cc:493] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:44.790752 30900 raft_consensus.cc:3060] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.794507 30900 raft_consensus.cc:515] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:44.797331 28983 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet e196dd3c7fe14524a50152643ddbc2ad (table default.Table_2dfebd62a146446f999f1a7a1bd1af79 [id=02ffed3d852845d6845fb5176f036fce], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e196dd3c7fe14524a50152643ddbc2ad consensus state has no leader
I20260618 13:59:44.798501 28617 leader_election.cc:304] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:44.799259 30854 raft_consensus.cc:2804] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:44.799356 30854 raft_consensus.cc:493] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:44.799440 30854 raft_consensus.cc:3060] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.802196 30854 raft_consensus.cc:515] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.803833 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b91f053bcc42bb829bb8180e232c8e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:44.804088 30854 leader_election.cc:290] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:44.804446 28618 leader_election.cc:343] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:44.805198 28483 leader_election.cc:343] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 8fea86ca38a741238a59053151087f69
I20260618 13:59:44.806654 28797 tablet_bootstrap.cc:654] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.806628 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b91f053bcc42bb829bb8180e232c8e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:44.806896 28408 raft_consensus.cc:3060] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.807508 30900 leader_election.cc:290] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:44.808365 28483 leader_election.cc:343] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 8fea86ca38a741238a59053151087f69
I20260618 13:59:44.808490 30856 consensus_queue.cc:1048] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:44.808934 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fea86ca38a741238a59053151087f69" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:44.809085 28682 raft_consensus.cc:3060] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.809609 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89b26d1595642ff82381f8baddfb99a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:44.809572 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "134a43259d5b429fbe47d0555d9942f3" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:44.811828 28682 raft_consensus.cc:2468] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:44.812371 28484 leader_election.cc:304] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:44.816051 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "134a43259d5b429fbe47d0555d9942f3" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:44.820274 30900 raft_consensus.cc:2804] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:44.821229 30900 raft_consensus.cc:697] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:44.821557 30900 consensus_queue.cc:237] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.833690 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet b0d3d363766546c4a97b0763bad11f71 with delete_type TABLET_DATA_DELETED (replaced by tablet 00b602a473a94c05a617fb9478300c74) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:44.834677 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 2128dbec33db453ba51070767a15b21e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:44.835520 28797 tablet_bootstrap.cc:492] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:44.835661 28797 ts_tablet_manager.cc:1403] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.075s user 0.002s sys 0.010s
I20260618 13:59:44.836210 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:44.836323 28797 raft_consensus.cc:359] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.836558 28797 raft_consensus.cc:385] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.836629 28797 raft_consensus.cc:740] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.836923 28797 consensus_queue.cc:260] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.837028 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet f2b21dcf74fa4264947ca43cc0026e76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:40 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:44.837549 28797 ts_tablet_manager.cc:1434] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:44.840440 28797 tablet_bootstrap.cc:492] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:44.842727 28346 leader_election.cc:343] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:44.846534 28346 leader_election.cc:343] T 134a43259d5b429fbe47d0555d9942f3 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.851123 28537 raft_consensus.cc:3060] T d0bcff62290c43c9be7cc496fbff6142 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.860522 29024 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_25a9b51a35344e3ca6689e9847de130c [id=f50e59ae00344758833392841b7f4985] alter complete (version 1)
I20260618 13:59:44.866667 28797 tablet_bootstrap.cc:654] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:44.868252 28646 tablet_service.cc:1558] Processing DeleteTablet for tablet e5f7eaa7d218440ca2c35340c5c87814 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.868418 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet 9c17e76ae9b44207998a43e3576908c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 2128dbec33db453ba51070767a15b21e) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.868458 28645 tablet_service.cc:1558] Processing DeleteTablet for tablet 283523e5b22843ba935eabe6283d3a99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.871023 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet 46bfe8cc2078414d84e65e22ed4ef8a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.871254 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet 962bf304a285412fb2a529db5b994e9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:34 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.871465 28548 raft_consensus.cc:3060] T c871e7c8f843486da7ba40145c0fed19 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:44.871513 28650 tablet_service.cc:1558] Processing DeleteTablet for tablet 04dac931e62a448d99633878eb70b92f with delete_type TABLET_DATA_DELETED (replaced by tablet 15483938a06e459789f87c6bb456bd15) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.871639 28659 tablet_service.cc:1558] Processing DeleteTablet for tablet d4667fe48f0f4eff9c6c8baabdab58d4 with delete_type TABLET_DATA_DELETED (replaced by tablet a662bb37fb7d4df49e61e524808b67a8) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.871809 28648 tablet_service.cc:1558] Processing DeleteTablet for tablet 55ae29c3b7ed44cfa659130745037aa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:31 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.871955 28651 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a0691de253b4b008e5d995fd6b876c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.872097 28658 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e25027f423144b088dad955c7282921 with delete_type TABLET_DATA_DELETED (replaced by tablet a8da96fc621c47d4a444b5a39259f85d) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.872226 28649 tablet_service.cc:1558] Processing DeleteTablet for tablet 6c4a31a4f202473d97b17b5b5dfdb615 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.872354 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet f45556a59c424ca2bc30956566374c89 with delete_type TABLET_DATA_DELETED (replaced by tablet 6903cb63c5b04408b7eaecf0c0231c42) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.872496 28663 tablet_service.cc:1558] Processing DeleteTablet for tablet cd06bb7cffc34412b420d70435fce373 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.872609 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet 35d8dd2c388e43a0be380aa953ec2c82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:34 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.872766 28646 tablet_service.cc:1558] Processing DeleteTablet for tablet d3dbf13544ce408ab97834aec5862390 with delete_type TABLET_DATA_DELETED (replaced by tablet ba18e879631947f88bd266681746ec82) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.872880 28645 tablet_service.cc:1558] Processing DeleteTablet for tablet 98b42d23832545ae81a7cdff19e4689f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:33 UTC) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:44.873292 29018 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet 2128dbec33db453ba51070767a15b21e (table default.Table_1af4cab3531c47b5a9cd4fee3f8251c2 [id=ef849a614b5040f79ced1b5abce5313e], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 2128dbec33db453ba51070767a15b21e consensus state has no leader
I20260618 13:59:44.874174 28644 tablet_service.cc:1558] Processing DeleteTablet for tablet 97a9f949999446f09359629fe483bf67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:29 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.874377 28663 tablet_service.cc:1558] Processing DeleteTablet for tablet 3dd0eaa64271489d92aa380df877f257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.875218 28646 tablet_service.cc:1558] Processing DeleteTablet for tablet ca2ee746c5024cfc9b31c470f9616e7d with delete_type TABLET_DATA_DELETED (replaced by tablet 60b6216474b74a14af6abf977e312b77) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.876986 28645 tablet_service.cc:1558] Processing DeleteTablet for tablet c7f42997416841a7b2c1ad347d2e9282 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.878273 28645 tablet_service.cc:1558] Processing DeleteTablet for tablet c4cfca9e76e641a7bc0b58b9ee240f36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.878628 28645 tablet_service.cc:1558] Processing DeleteTablet for tablet e3d8616846c649b6a969b971db0d03db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.845824 28408 raft_consensus.cc:2468] T 51b91f053bcc42bb829bb8180e232c8e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:44.885617 28617 leader_election.cc:304] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:44.888343 30856 raft_consensus.cc:2804] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:44.888475 30856 raft_consensus.cc:697] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:44.888698 30856 consensus_queue.cc:237] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.889819 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 261) took 3817 ms. Trace:
I20260618 13:59:44.894594 30900 raft_consensus.cc:493] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:44.894765 30900 raft_consensus.cc:515] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:44.895474 30900 leader_election.cc:290] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:44.895658 28528 tablet_service.cc:1511] Processing CreateTablet for tablet 8ac84641d0f1422193e42a42468789e1 (DEFAULT_TABLE table=default.Table_e98f724cea1e4b7c8acd2a357fb845a0 [id=95cb33ea8eef404e87348fda4b2f1805]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:44.895653 29566 rpcz_store.cc:276] 0618 13:59:41.072031 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.072367 (+ 336us) service_pool.cc:224] Handling call
0618 13:59:44.530616 (+3458249us) ts_tablet_manager.cc:1151] Deleting tablet f2b21dcf74fa4264947ca43cc0026e76
0618 13:59:44.530628 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.613754 (+ 83126us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:44.787309 (+173555us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:44.889803 (+102494us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3443239}
I20260618 13:59:44.896179 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ac84641d0f1422193e42a42468789e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:44.896130 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet f2b21dcf74fa4264947ca43cc0026e76 (table default.Table_c35470a550534fa2ad072f932e981df5 [id=2797c7fefad74c1e82ab31fb9ad4db70]) successfully deleted
I20260618 13:59:44.896482 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 284) took 3746 ms. Trace:
I20260618 13:59:44.896588 29566 rpcz_store.cc:276] 0618 13:59:41.149965 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.150021 (+ 56us) service_pool.cc:224] Handling call
0618 13:59:44.896395 (+3746374us) ts_tablet_manager.cc:1151] Deleting tablet 2f3d2f236fd44ab8af0524ea282cfa94
0618 13:59:44.896404 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.896465 (+ 61us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3746177}
W20260618 13:59:44.896797 28483 leader_election.cc:343] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 98446f40214c4ac984fcfe98310005c5
W20260618 13:59:44.897009 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 2f3d2f236fd44ab8af0524ea282cfa94 because the tablet was not found. No further retry: Not found: Tablet not found: 2f3d2f236fd44ab8af0524ea282cfa94
W20260618 13:59:44.900269 28484 leader_election.cc:343] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 98446f40214c4ac984fcfe98310005c5
I20260618 13:59:44.900400 28484 leader_election.cc:304] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:44.900791 30900 raft_consensus.cc:2749] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:44.907747 28645 tablet_service.cc:1511] Processing CreateTablet for tablet 8ac84641d0f1422193e42a42468789e1 (DEFAULT_TABLE table=default.Table_e98f724cea1e4b7c8acd2a357fb845a0 [id=95cb33ea8eef404e87348fda4b2f1805]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:44.908270 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 287) took 3746 ms. Trace:
I20260618 13:59:44.908406 28645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ac84641d0f1422193e42a42468789e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:44.908357 29566 rpcz_store.cc:276] 0618 13:59:41.162211 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.162255 (+ 44us) service_pool.cc:224] Handling call
0618 13:59:44.899656 (+3737401us) ts_tablet_manager.cc:1151] Deleting tablet c552d1316e3340f4bcfa39618ae309b8
0618 13:59:44.908143 (+ 8487us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.908250 (+ 107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13696,"tablet-delete.queue_time_us":3737200}
W20260618 13:59:44.909541 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet c552d1316e3340f4bcfa39618ae309b8 because the tablet was not found. No further retry: Not found: Tablet not found: c552d1316e3340f4bcfa39618ae309b8
I20260618 13:59:44.909859 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 288) took 3747 ms. Trace:
I20260618 13:59:44.909938 29566 rpcz_store.cc:276] 0618 13:59:41.162630 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.162667 (+ 37us) service_pool.cc:224] Handling call
0618 13:59:44.909777 (+3747110us) ts_tablet_manager.cc:1151] Deleting tablet 75824a5d2f7e47838b60e6a2b785c6b2
0618 13:59:44.909787 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.909844 (+ 57us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3746982}
W20260618 13:59:44.910293 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 75824a5d2f7e47838b60e6a2b785c6b2 because the tablet was not found. No further retry: Not found: Tablet not found: 75824a5d2f7e47838b60e6a2b785c6b2
I20260618 13:59:44.912216 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 289) took 3749 ms. Trace:
I20260618 13:59:44.912305 29566 rpcz_store.cc:276] 0618 13:59:41.162968 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.163004 (+ 36us) service_pool.cc:224] Handling call
0618 13:59:44.912121 (+3749117us) ts_tablet_manager.cc:1151] Deleting tablet 827cc0987bd541759e334fb620f04ed5
0618 13:59:44.912133 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.912200 (+ 67us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3748991}
W20260618 13:59:44.912761 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 827cc0987bd541759e334fb620f04ed5 because the tablet was not found. No further retry: Not found: Tablet not found: 827cc0987bd541759e334fb620f04ed5
I20260618 13:59:44.913079 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 290) took 3749 ms. Trace:
I20260618 13:59:44.913156 29566 rpcz_store.cc:276] 0618 13:59:41.163319 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.163356 (+ 37us) service_pool.cc:224] Handling call
0618 13:59:44.912995 (+3749639us) ts_tablet_manager.cc:1151] Deleting tablet 864f2602840c40fc8606b3784e671cba
0618 13:59:44.913004 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.913062 (+ 58us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3749522}
W20260618 13:59:44.913561 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 864f2602840c40fc8606b3784e671cba because the tablet was not found. No further retry: Not found: Tablet not found: 864f2602840c40fc8606b3784e671cba
I20260618 13:59:44.914940 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 291) took 3750 ms. Trace:
I20260618 13:59:44.915019 29566 rpcz_store.cc:276] 0618 13:59:41.164206 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.164246 (+ 40us) service_pool.cc:224] Handling call
0618 13:59:44.914862 (+3750616us) ts_tablet_manager.cc:1151] Deleting tablet 591465e2b57247a2b224cc8b09225811
0618 13:59:44.914872 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.914924 (+ 52us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3750466}
W20260618 13:59:44.915436 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 591465e2b57247a2b224cc8b09225811 because the tablet was not found. No further retry: Not found: Tablet not found: 591465e2b57247a2b224cc8b09225811
W20260618 13:59:44.915982 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet b0d3d363766546c4a97b0763bad11f71 because the tablet was not found. No further retry: Not found: Tablet not found: b0d3d363766546c4a97b0763bad11f71
W20260618 13:59:44.916517 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 2128dbec33db453ba51070767a15b21e because the tablet was not found. No further retry: Not found: Tablet not found: 2128dbec33db453ba51070767a15b21e
W20260618 13:59:44.916851 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 because the tablet was not found. No further retry: Not found: Tablet not found: 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3
W20260618 13:59:44.917730 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet f2b21dcf74fa4264947ca43cc0026e76 because the tablet was not found. No further retry: Not found: Tablet not found: f2b21dcf74fa4264947ca43cc0026e76
W20260618 13:59:44.920253 29018 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet 72c04f55890648f0a8184f3783243f7c (table default.Table_773be314d7e741fcb4fb6eb7d954802e [id=0d83fc68255c458587390c58b12e00d7], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 72c04f55890648f0a8184f3783243f7c consensus state has no leader
W20260618 13:59:44.923811 29018 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.3:42065 (request call id 148) took 5239 ms (client timeout 1000 ms). Trace:
W20260618 13:59:44.923979 29018 rpcz_store.cc:269] 0618 13:59:39.684022 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:40.240806 (+556784us) service_pool.cc:224] Handling call
0618 13:59:44.923753 (+4682947us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:41.169476 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:41.169624 (+ 148us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:41.169634 (+ 10us) write_op.cc:435] Acquired schema lock
0618 13:59:41.169638 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:41.169855 (+ 217us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:41.169874 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:41.169883 (+ 9us) tablet.cc:685] Acquiring locks for 3 operations
0618 13:59:41.179361 (+ 9478us) tablet.cc:701] Row locks acquired
0618 13:59:41.179364 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:41.179407 (+ 43us) write_op.cc:270] Start()
0618 13:59:41.179443 (+ 36us) write_op.cc:276] Timestamp: P: 1781791181179357 usec, L: 0
0618 13:59:41.179447 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:41.179927 (+ 480us) log.cc:844] Serialized 1284 byte log entry
0618 13:59:41.220154 (+ 40227us) op_driver.cc:464] REPLICATION: finished
0618 13:59:41.220284 (+ 130us) write_op.cc:301] APPLY: starting
0618 13:59:41.220302 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:41.220365 (+ 63us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:41.220391 (+ 26us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:41.220471 (+ 80us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:41.220504 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0618 13:59:41.220512 (+ 8us) write_op.cc:312] APPLY: finished
0618 13:59:41.220594 (+ 82us) log.cc:844] Serialized 35 byte log entry
0618 13:59:41.220655 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.220677 (+ 22us) write_op.cc:454] Released schema lock
0618 13:59:41.220684 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:41.220755 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:41.220759 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":68238,"reported_tablets":184,"child_traces":[["op",{"apply.queue_time_us":82,"apply.run_cpu_time_us":462,"apply.run_wall_time_us":461,"num_ops":3,"prepare.queue_time_us":49,"prepare.run_cpu_time_us":1067,"prepare.run_wall_time_us":10570,"raft.queue_time_us":739,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":205,"replication_time_us":40674}]]}
I20260618 13:59:44.926851 28645 tablet_service.cc:1558] Processing DeleteTablet for tablet f278d0acbb0c4a0ca61640bc142e6335 with delete_type TABLET_DATA_DELETED (replaced by tablet 72c04f55890648f0a8184f3783243f7c) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.927053 28646 tablet_service.cc:1558] Processing DeleteTablet for tablet e6de1293572f4e75af3417f281b9a142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.927337 28646 tablet_service.cc:1558] Processing DeleteTablet for tablet 827cc0987bd541759e334fb620f04ed5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.927831 28649 tablet_service.cc:1558] Processing DeleteTablet for tablet be59f988f0e34b9480f859cf88b9c223 with delete_type TABLET_DATA_DELETED (replaced by tablet b968a8d4014341ca9c5964263721ac79) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.928288 28649 tablet_service.cc:1558] Processing DeleteTablet for tablet 015494374645469785f1f978eaf0e9a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 2e493f647a344cd581515a77230d765f) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.928462 28660 tablet_service.cc:1558] Processing DeleteTablet for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 with delete_type TABLET_DATA_DELETED (replaced by tablet cea4bc13eabc4dd98b4f96723498014c) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.928526 28646 tablet_service.cc:1558] Processing DeleteTablet for tablet ac58e939730c4af681c2f08bcfbb6602 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:22 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.928663 28653 tablet_service.cc:1558] Processing DeleteTablet for tablet cea4bc13eabc4dd98b4f96723498014c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.928824 28658 tablet_service.cc:1558] Processing DeleteTablet for tablet 864f2602840c40fc8606b3784e671cba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.928966 28663 tablet_service.cc:1558] Processing DeleteTablet for tablet 591465e2b57247a2b224cc8b09225811 with delete_type TABLET_DATA_DELETED (replaced by tablet d4667fe48f0f4eff9c6c8baabdab58d4) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.929117 28644 tablet_service.cc:1558] Processing DeleteTablet for tablet c3d523419ff244f085d6eac1a3231973 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.929253 28651 tablet_service.cc:1558] Processing DeleteTablet for tablet f3ba9186b3ff47d5a2b158fef619686e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.929400 28648 tablet_service.cc:1558] Processing DeleteTablet for tablet 98446f40214c4ac984fcfe98310005c5 with delete_type TABLET_DATA_DELETED (replaced by tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.929756 28648 tablet_service.cc:1558] Processing DeleteTablet for tablet c01c8463055e4526b3c75190066f30b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.930516 28651 tablet_service.cc:1558] Processing DeleteTablet for tablet 7572d17724d249739be1a1cb9468f909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:27 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.930925 28651 tablet_service.cc:1558] Processing DeleteTablet for tablet ab521028ac1a4d0da013112d76bb169f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:44.928674 28645 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fae928833e44d2ba860d6184d4c0142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:44.923827 29023 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.3:42065 (request call id 149) took 3832 ms (client timeout 1000 ms). Trace:
W20260618 13:59:44.932998 29023 rpcz_store.cc:269] 0618 13:59:41.091458 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.110782 (+ 19324us) service_pool.cc:224] Handling call
0618 13:59:44.923766 (+3812984us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:42.439242 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:42.439429 (+ 187us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:42.439439 (+ 10us) write_op.cc:435] Acquired schema lock
0618 13:59:42.439444 (+ 5us) tablet.cc:662] Decoding operations
0618 13:59:42.441028 (+ 1584us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:42.441044 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:42.441050 (+ 6us) tablet.cc:685] Acquiring locks for 6 operations
0618 13:59:42.441256 (+ 206us) tablet.cc:701] Row locks acquired
0618 13:59:42.441259 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:42.441307 (+ 48us) write_op.cc:270] Start()
0618 13:59:42.441348 (+ 41us) write_op.cc:276] Timestamp: P: 1781791182441283 usec, L: 0
0618 13:59:42.441353 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:42.441622 (+ 269us) log.cc:844] Serialized 2416 byte log entry
0618 13:59:42.502443 (+ 60821us) op_driver.cc:464] REPLICATION: finished
0618 13:59:42.509053 (+ 6610us) write_op.cc:301] APPLY: starting
0618 13:59:42.509076 (+ 23us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:42.509135 (+ 59us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:42.509179 (+ 44us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:42.509330 (+ 151us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:42.509371 (+ 41us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0618 13:59:42.509379 (+ 8us) write_op.cc:312] APPLY: finished
0618 13:59:42.509500 (+ 121us) log.cc:844] Serialized 53 byte log entry
0618 13:59:42.515830 (+ 6330us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:42.515865 (+ 35us) write_op.cc:454] Released schema lock
0618 13:59:42.515876 (+ 11us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:42.534942 (+ 19066us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:42.534955 (+ 13us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:42.528033 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:42.528162 (+ 129us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:42.528169 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:42.528171 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:42.528308 (+ 137us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:42.528324 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:42.528785 (+ 461us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:42.528908 (+ 123us) tablet.cc:701] Row locks acquired
0618 13:59:42.528912 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:42.528958 (+ 46us) write_op.cc:270] Start()
0618 13:59:42.528999 (+ 41us) write_op.cc:276] Timestamp: P: 1781791182528925 usec, L: 0
0618 13:59:42.529003 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:42.529210 (+ 207us) log.cc:844] Serialized 356 byte log entry
0618 13:59:42.620601 (+ 91391us) op_driver.cc:464] REPLICATION: finished
0618 13:59:42.632428 (+ 11827us) write_op.cc:301] APPLY: starting
0618 13:59:42.632452 (+ 24us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:42.632500 (+ 48us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:42.632536 (+ 36us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:42.632602 (+ 66us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:42.632640 (+ 38us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:42.632647 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:42.632744 (+ 97us) log.cc:844] Serialized 23 byte log entry
0618 13:59:42.632826 (+ 82us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:42.632851 (+ 25us) write_op.cc:454] Released schema lock
0618 13:59:42.632859 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:42.632942 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:42.632946 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:43.084700 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:43.084813 (+ 113us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:43.084818 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:43.084820 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:43.084903 (+ 83us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:43.084911 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:43.084915 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:43.084979 (+ 64us) tablet.cc:701] Row locks acquired
0618 13:59:43.084981 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:43.085013 (+ 32us) write_op.cc:270] Start()
0618 13:59:43.085038 (+ 25us) write_op.cc:276] Timestamp: P: 1781791183084982 usec, L: 0
0618 13:59:43.085043 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:43.121816 (+ 36773us) log.cc:844] Serialized 356 byte log entry
0618 13:59:43.122015 (+ 199us) spinlock_profiling.cc:243] Waited 36.6 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fcbac75ef22 00007fcbadb40305 00007fcbadb3ee29 00007fcbadb47250 00007fcbadb4705d 0000000000aa681c 00007fcba07299ff 00007fcba072cd1b 00007fcba072cb98 0000000000aa681c
0618 13:59:43.172777 (+ 50762us) op_driver.cc:464] REPLICATION: finished
0618 13:59:43.173146 (+ 369us) write_op.cc:301] APPLY: starting
0618 13:59:43.173161 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:43.173205 (+ 44us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:43.173207 (+ 2us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:43.173253 (+ 46us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:43.173282 (+ 29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:43.173288 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:43.173354 (+ 66us) log.cc:844] Serialized 23 byte log entry
0618 13:59:43.173408 (+ 54us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:43.173423 (+ 15us) write_op.cc:454] Released schema lock
0618 13:59:43.173429 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:43.173508 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:43.173512 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:44.703310 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:44.703478 (+ 168us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:44.703488 (+ 10us) write_op.cc:435] Acquired schema lock
0618 13:59:44.703492 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:44.703630 (+ 138us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:44.703648 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:44.703655 (+ 7us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:44.703759 (+ 104us) tablet.cc:701] Row locks acquired
0618 13:59:44.703764 (+ 5us) write_op.cc:260] PREPARE: finished
0618 13:59:44.703807 (+ 43us) write_op.cc:270] Start()
0618 13:59:44.703853 (+ 46us) write_op.cc:276] Timestamp: P: 1781791184703760 usec, L: 0
0618 13:59:44.703858 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:44.704160 (+ 302us) log.cc:844] Serialized 356 byte log entry
0618 13:59:44.711706 (+ 7546us) op_driver.cc:464] REPLICATION: finished
0618 13:59:44.712018 (+ 312us) write_op.cc:301] APPLY: starting
0618 13:59:44.712038 (+ 20us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:44.712084 (+ 46us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:44.715584 (+ 3500us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:44.715674 (+ 90us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:44.715719 (+ 45us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:44.715731 (+ 12us) write_op.cc:312] APPLY: finished
0618 13:59:44.715825 (+ 94us) log.cc:844] Serialized 23 byte log entry
0618 13:59:44.715929 (+ 104us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.715953 (+ 24us) write_op.cc:454] Released schema lock
0618 13:59:44.715962 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:44.716056 (+ 94us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.716061 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":180071,"reported_tablets":177,"spinlock_wait_cycles":15616,"child_traces":[["op",{"apply.queue_time_us":6520,"apply.run_cpu_time_us":733,"apply.run_wall_time_us":25743,"num_ops":6,"prepare.queue_time_us":11315,"prepare.run_cpu_time_us":1416,"prepare.run_wall_time_us":13394,"raft.queue_time_us":92,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":346,"replication_time_us":61031,"thread_start_us":19691,"threads_started":2}],["op",{"apply.queue_time_us":11748,"apply.run_cpu_time_us":514,"apply.run_wall_time_us":513,"num_ops":1,"prepare.queue_time_us":81,"prepare.run_cpu_time_us":883,"prepare.run_wall_time_us":1772,"raft.queue_time_us":4866,"raft.run_cpu_time_us":12,"raft.run_wall_time_us":13,"replication_time_us":91559}],["op",{"apply.queue_time_us":313,"apply.run_cpu_time_us":345,"apply.run_wall_time_us":345,"num_ops":1,"prepare.queue_time_us":44,"prepare.run_cpu_time_us":1418,"prepare.run_wall_time_us":37370,"raft.queue_time_us":795,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":87702,"spinlock_wait_cycles":84115712}],["op",{"apply.queue_time_us":257,"apply.run_cpu_time_us":584,"apply.run_wall_time_us":4022,"num_ops":1,"prepare.queue_time_us":13924,"prepare.run_cpu_time_us":1002,"prepare.run_wall_time_us":1003,"raft.queue_time_us":2735,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":344,"replication_time_us":7816}]]}
I20260618 13:59:44.934641 30892 tablet.cc:1724] T 10e9a417eecf4d33ac3bd9bd6325f447 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260618 13:59:44.958132 29026 catalog_manager.cc:5888] Failed to send AlterTable RPC for tablet 00b602a473a94c05a617fb9478300c74 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 00b602a473a94c05a617fb9478300c74 consensus state has no leader
I20260618 13:59:44.959208 28645 tablet_service.cc:1558] Processing DeleteTablet for tablet f2b21dcf74fa4264947ca43cc0026e76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:40 UTC) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:44.960152 29026 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.3:42065 (request call id 152) took 1768 ms (client timeout 1000 ms). Trace:
W20260618 13:59:44.960284 29026 rpcz_store.cc:269] 0618 13:59:43.191796 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:43.733647 (+541851us) service_pool.cc:224] Handling call
0618 13:59:44.960114 (+1226467us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:44.458757 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:44.458911 (+ 154us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:44.458919 (+ 8us) write_op.cc:435] Acquired schema lock
0618 13:59:44.458922 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:44.459130 (+ 208us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:44.459144 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:44.459149 (+ 5us) tablet.cc:685] Acquiring locks for 5 operations
0618 13:59:44.459324 (+ 175us) tablet.cc:701] Row locks acquired
0618 13:59:44.459327 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:44.459365 (+ 38us) write_op.cc:270] Start()
0618 13:59:44.459401 (+ 36us) write_op.cc:276] Timestamp: P: 1781791184459346 usec, L: 0
0618 13:59:44.459405 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:44.459633 (+ 228us) log.cc:844] Serialized 2039 byte log entry
0618 13:59:44.487239 (+ 27606us) op_driver.cc:464] REPLICATION: finished
0618 13:59:44.487432 (+ 193us) write_op.cc:301] APPLY: starting
0618 13:59:44.487454 (+ 22us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:44.487566 (+ 112us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:44.487611 (+ 45us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:44.487831 (+ 220us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:44.487873 (+ 42us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0618 13:59:44.487881 (+ 8us) write_op.cc:312] APPLY: finished
0618 13:59:44.487968 (+ 87us) log.cc:844] Serialized 47 byte log entry
0618 13:59:44.488033 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.488059 (+ 26us) write_op.cc:454] Released schema lock
0618 13:59:44.488066 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:44.488158 (+ 92us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.488164 (+ 6us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:44.671856 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:44.672012 (+ 156us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:44.672022 (+ 10us) write_op.cc:435] Acquired schema lock
0618 13:59:44.672026 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:44.672149 (+ 123us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:44.672167 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:44.672175 (+ 8us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:44.672257 (+ 82us) tablet.cc:701] Row locks acquired
0618 13:59:44.672261 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:44.672303 (+ 42us) write_op.cc:270] Start()
0618 13:59:44.672345 (+ 42us) write_op.cc:276] Timestamp: P: 1781791184672258 usec, L: 0
0618 13:59:44.672350 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:44.672575 (+ 225us) log.cc:844] Serialized 356 byte log entry
0618 13:59:44.676241 (+ 3666us) op_driver.cc:464] REPLICATION: finished
0618 13:59:44.676361 (+ 120us) write_op.cc:301] APPLY: starting
0618 13:59:44.676377 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:44.676419 (+ 42us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:44.676444 (+ 25us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:44.676496 (+ 52us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:44.676526 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:44.676533 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:44.676612 (+ 79us) log.cc:844] Serialized 23 byte log entry
0618 13:59:44.676670 (+ 58us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.676688 (+ 18us) write_op.cc:454] Released schema lock
0618 13:59:44.676694 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:44.676765 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.676769 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":42343,"reported_tablets":163,"child_traces":[["op",{"apply.queue_time_us":129,"apply.run_cpu_time_us":711,"apply.run_wall_time_us":710,"num_ops":5,"prepare.queue_time_us":20760,"prepare.run_cpu_time_us":1004,"prepare.run_wall_time_us":1004,"raft.queue_time_us":805,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":268,"replication_time_us":27789}],["op",{"apply.queue_time_us":78,"apply.run_cpu_time_us":391,"apply.run_wall_time_us":390,"num_ops":1,"prepare.queue_time_us":63,"prepare.run_cpu_time_us":849,"prepare.run_wall_time_us":849,"raft.queue_time_us":930,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":308,"replication_time_us":3862}]]}
I20260618 13:59:44.961417 29026 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_8c9c9a6eea174bd68d8fb9d9ebaaf603" } modify_external_catalogs: true
I20260618 13:59:44.961761 29026 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_8c9c9a6eea174bd68d8fb9d9ebaaf603" } modify_external_catalogs: true
W20260618 13:59:44.964558 29011 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.1:48495 (request call id 151) took 3799 ms (client timeout 1000 ms). Trace:
W20260618 13:59:44.967444 28346 leader_election.cc:343] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:44.967581 28346 leader_election.cc:304] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:44.970824 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 292) took 3797 ms. Trace:
I20260618 13:59:44.970925 29566 rpcz_store.cc:276] 0618 13:59:41.173697 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.173931 (+ 234us) service_pool.cc:224] Handling call
0618 13:59:44.970688 (+3796757us) ts_tablet_manager.cc:1151] Deleting tablet 02b5cc08ca0c4e719e0414b345211ba5
0618 13:59:44.970704 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.970797 (+ 93us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3796567}
W20260618 13:59:44.971371 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 02b5cc08ca0c4e719e0414b345211ba5 because the tablet was not found. No further retry: Not found: Tablet not found: 02b5cc08ca0c4e719e0414b345211ba5
I20260618 13:59:44.971464 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 293) took 3797 ms. Trace:
I20260618 13:59:44.971586 29566 rpcz_store.cc:276] 0618 13:59:41.174304 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.174849 (+ 545us) service_pool.cc:224] Handling call
0618 13:59:44.971379 (+3796530us) ts_tablet_manager.cc:1151] Deleting tablet 6eea4d59935d40aa97dc10ec7469aa56
0618 13:59:44.971388 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.971446 (+ 58us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3796351}
W20260618 13:59:44.972160 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 6eea4d59935d40aa97dc10ec7469aa56 because the tablet was not found. No further retry: Not found: Tablet not found: 6eea4d59935d40aa97dc10ec7469aa56
I20260618 13:59:44.973122 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 304) took 3701 ms. Trace:
I20260618 13:59:44.973199 29566 rpcz_store.cc:276] 0618 13:59:41.271915 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.272195 (+ 280us) service_pool.cc:224] Handling call
0618 13:59:44.973051 (+3700856us) ts_tablet_manager.cc:1151] Deleting tablet 8b59255b14df4158afe37cb8890e7415
0618 13:59:44.973059 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:44.973108 (+ 49us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3649415}
W20260618 13:59:44.973572 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 8b59255b14df4158afe37cb8890e7415 because the tablet was not found. No further retry: Not found: Tablet not found: 8b59255b14df4158afe37cb8890e7415
I20260618 13:59:44.981053 30947 tablet.cc:1724] T 10e9a417eecf4d33ac3bd9bd6325f447 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:44.986395 28798 tablet_bootstrap.cc:492] T 5bbb71449d434d54b007d6f0da0796d7 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:44.986549 28798 ts_tablet_manager.cc:1403] T 5bbb71449d434d54b007d6f0da0796d7 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.269s user 0.031s sys 0.052s
I20260618 13:59:44.987221 28798 raft_consensus.cc:359] T 5bbb71449d434d54b007d6f0da0796d7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.987381 28798 raft_consensus.cc:385] T 5bbb71449d434d54b007d6f0da0796d7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:44.987440 28798 raft_consensus.cc:740] T 5bbb71449d434d54b007d6f0da0796d7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:44.987612 28798 consensus_queue.cc:260] T 5bbb71449d434d54b007d6f0da0796d7 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:44.988087 28798 ts_tablet_manager.cc:1434] T 5bbb71449d434d54b007d6f0da0796d7 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 13:59:44.988554 28798 tablet_bootstrap.cc:492] T d4539f767c054e9e8ebc1f3f56108e7e P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:44.992944 28798 tablet_bootstrap.cc:654] T d4539f767c054e9e8ebc1f3f56108e7e P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.000666 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 330) took 3706 ms. Trace:
I20260618 13:59:45.000854 29566 rpcz_store.cc:276] 0618 13:59:41.294238 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.323654 (+ 29416us) service_pool.cc:224] Handling call
0618 13:59:44.999451 (+3675797us) ts_tablet_manager.cc:1151] Deleting tablet ba2d6894bf5649fc8cfae7e033c432b9
0618 13:59:45.000541 (+ 1090us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.000644 (+ 103us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3675569}
I20260618 13:59:45.001032 30847 raft_consensus.cc:493] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.002840 30847 raft_consensus.cc:515] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.003607 30847 leader_election.cc:290] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.002171 30854 consensus_queue.cc:1048] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:45.005568 28617 leader_election.cc:343] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 72c04f55890648f0a8184f3783243f7c
I20260618 13:59:45.005577 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet c5d046e0421142cd84c5b12d6a434f53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:43 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.005925 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 98446f40214c4ac984fcfe98310005c5 with delete_type TABLET_DATA_DELETED (replaced by tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.006130 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet c01c8463055e4526b3c75190066f30b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.006246 28798 tablet_bootstrap.cc:492] T d4539f767c054e9e8ebc1f3f56108e7e P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:45.006369 28798 ts_tablet_manager.cc:1403] T d4539f767c054e9e8ebc1f3f56108e7e P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.006s
I20260618 13:59:45.006630 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet e3d8616846c649b6a969b971db0d03db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.006660 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 302) took 3734 ms. Trace:
I20260618 13:59:45.006732 29566 rpcz_store.cc:276] 0618 13:59:41.271738 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.272655 (+ 917us) service_pool.cc:224] Handling call
0618 13:59:45.006382 (+3733727us) ts_tablet_manager.cc:1151] Deleting tablet 18d8b609b96a4e81b57f69bff522eeea
0618 13:59:45.006393 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.006642 (+ 249us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3682424}
I20260618 13:59:45.007680 28798 raft_consensus.cc:359] T d4539f767c054e9e8ebc1f3f56108e7e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.007838 28798 raft_consensus.cc:385] T d4539f767c054e9e8ebc1f3f56108e7e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260618 13:59:45.007828 28989 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet ba2d6894bf5649fc8cfae7e033c432b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba2d6894bf5649fc8cfae7e033c432b9 already in progress: creating tablet
I20260618 13:59:45.007897 28798 raft_consensus.cc:740] T d4539f767c054e9e8ebc1f3f56108e7e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.008061 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 305) took 3736 ms. Trace:
I20260618 13:59:45.008109 28798 consensus_queue.cc:260] T d4539f767c054e9e8ebc1f3f56108e7e P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:45.008280 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 18d8b609b96a4e81b57f69bff522eeea because the tablet was not found. No further retry: Not found: Tablet not found: 18d8b609b96a4e81b57f69bff522eeea
I20260618 13:59:45.008821 28798 ts_tablet_manager.cc:1434] T d4539f767c054e9e8ebc1f3f56108e7e P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:45.008899 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet fea86899d2c047d997cebe80551e83c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.009088 28798 tablet_bootstrap.cc:492] T e3a3395aa90e426eb48a3920a8c45ba8 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:45.009486 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 8fea86ca38a741238a59053151087f69 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f9ec5f31f6c4d37b7c3ec585ed3211b) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.009984 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 97a9f949999446f09359629fe483bf67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:29 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.010515 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet e5f7eaa7d218440ca2c35340c5c87814 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.008126 29566 rpcz_store.cc:276] 0618 13:59:41.272015 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.273050 (+ 1035us) service_pool.cc:224] Handling call
0618 13:59:45.007981 (+3734931us) ts_tablet_manager.cc:1151] Deleting tablet b51d0851efb34eaa9e9e4be6b292ef62
0618 13:59:45.007990 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.008046 (+ 56us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3683974}
I20260618 13:59:45.011191 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet b0d3d363766546c4a97b0763bad11f71 with delete_type TABLET_DATA_DELETED (replaced by tablet 00b602a473a94c05a617fb9478300c74) from {username='slave'} at 127.0.0.1:57282
W20260618 13:59:45.011520 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet b51d0851efb34eaa9e9e4be6b292ef62 because the tablet was not found. No further retry: Not found: Tablet not found: b51d0851efb34eaa9e9e4be6b292ef62
I20260618 13:59:45.011561 28798 tablet_bootstrap.cc:654] T e3a3395aa90e426eb48a3920a8c45ba8 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.012540 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 309) took 3730 ms. Trace:
I20260618 13:59:45.012653 29566 rpcz_store.cc:276] 0618 13:59:41.282452 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.282519 (+ 67us) service_pool.cc:224] Handling call
0618 13:59:45.012441 (+3729922us) ts_tablet_manager.cc:1151] Deleting tablet fddea8fba4da40d7a5780e6750caf5e9
0618 13:59:45.012450 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.012523 (+ 73us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3688162}
W20260618 13:59:45.013065 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet fddea8fba4da40d7a5780e6750caf5e9 because the tablet was not found. No further retry: Not found: Tablet not found: fddea8fba4da40d7a5780e6750caf5e9
I20260618 13:59:45.017508 28797 tablet_bootstrap.cc:492] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:45.017630 28797 ts_tablet_manager.cc:1403] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.179s user 0.029s sys 0.041s
I20260618 13:59:45.018247 28797 raft_consensus.cc:359] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.018399 28797 raft_consensus.cc:385] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.018474 28797 raft_consensus.cc:740] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.018639 28797 consensus_queue.cc:260] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:45.018702 29020 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.1:48495 (request call id 153) took 2349 ms (client timeout 1000 ms). Trace:
I20260618 13:59:45.019124 28797 ts_tablet_manager.cc:1434] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 13:59:45.019381 28797 tablet_bootstrap.cc:492] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:45.020705 28797 tablet_bootstrap.cc:654] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.021638 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.021970 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 2128dbec33db453ba51070767a15b21e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.022282 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet ba2d6894bf5649fc8cfae7e033c432b9 with delete_type TABLET_DATA_DELETED (replaced by tablet a6ba4976bb6e4c2da3a563ccad4eb2b0) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.022607 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 6c4a31a4f202473d97b17b5b5dfdb615 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.022924 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 72c04f55890648f0a8184f3783243f7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.023376 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 962bf304a285412fb2a529db5b994e9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:34 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.024258 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet f45556a59c424ca2bc30956566374c89 with delete_type TABLET_DATA_DELETED (replaced by tablet 6903cb63c5b04408b7eaecf0c0231c42) from {username='slave'} at 127.0.0.1:57282
W20260618 13:59:45.025126 29024 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.2:52853 (request call id 127) took 1352 ms (client timeout 1000 ms). Trace:
I20260618 13:59:45.026749 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet d4667fe48f0f4eff9c6c8baabdab58d4 with delete_type TABLET_DATA_DELETED (replaced by tablet a662bb37fb7d4df49e61e524808b67a8) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.026845 28797 tablet_bootstrap.cc:492] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:45.026988 28797 ts_tablet_manager.cc:1403] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260618 13:59:45.027153 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet f2b21dcf74fa4264947ca43cc0026e76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:40 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.027822 28797 raft_consensus.cc:359] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.028048 28797 raft_consensus.cc:385] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.028118 28797 raft_consensus.cc:740] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.028369 28797 consensus_queue.cc:260] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:45.018796 29020 rpcz_store.cc:269] 0618 13:59:42.669287 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:43.035570 (+366283us) service_pool.cc:224] Handling call
0618 13:59:45.018651 (+1983081us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:43.886807 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:43.888471 (+ 1664us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:43.888483 (+ 12us) write_op.cc:435] Acquired schema lock
0618 13:59:43.888488 (+ 5us) tablet.cc:662] Decoding operations
0618 13:59:43.888742 (+ 254us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:43.888758 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:43.888767 (+ 9us) tablet.cc:685] Acquiring locks for 4 operations
0618 13:59:43.888960 (+ 193us) tablet.cc:701] Row locks acquired
0618 13:59:43.888965 (+ 5us) write_op.cc:260] PREPARE: finished
0618 13:59:43.889015 (+ 50us) write_op.cc:270] Start()
0618 13:59:43.889062 (+ 47us) write_op.cc:276] Timestamp: P: 1781791183888929 usec, L: 0
0618 13:59:43.889068 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:43.889324 (+ 256us) log.cc:844] Serialized 1662 byte log entry
0618 13:59:44.026767 (+137443us) op_driver.cc:464] REPLICATION: finished
0618 13:59:44.027205 (+ 438us) write_op.cc:301] APPLY: starting
0618 13:59:44.027219 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:44.027286 (+ 67us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:44.027353 (+ 67us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:44.027457 (+ 104us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:44.027495 (+ 38us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0618 13:59:44.027502 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:44.027589 (+ 87us) log.cc:844] Serialized 41 byte log entry
0618 13:59:44.028307 (+ 718us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.028324 (+ 17us) write_op.cc:454] Released schema lock
0618 13:59:44.028332 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:44.043650 (+ 15318us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.043655 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":102720,"reported_tablets":138,"child_traces":[["op",{"apply.queue_time_us":375,"apply.run_cpu_time_us":552,"apply.run_wall_time_us":16433,"num_ops":4,"prepare.queue_time_us":46719,"prepare.run_cpu_time_us":1233,"prepare.run_wall_time_us":3171,"raft.queue_time_us":65,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":137672,"thread_start_us":1250,"threads_started":1}]]}
I20260618 13:59:45.028882 28797 ts_tablet_manager.cc:1434] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:45.029134 28797 tablet_bootstrap.cc:492] T 779755a607bf46219900412b527e58b5 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:45.027384 29024 rpcz_store.cc:269] 0618 13:59:43.673115 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:43.798539 (+125424us) service_pool.cc:224] Handling call
0618 13:59:44.246936 (+448397us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.246941 (+ 5us) write_op.cc:454] Released schema lock
0618 13:59:45.025070 (+778129us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:44.239622 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:44.239761 (+ 139us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:44.239771 (+ 10us) write_op.cc:435] Acquired schema lock
0618 13:59:44.239776 (+ 5us) tablet.cc:662] Decoding operations
0618 13:59:44.239894 (+ 118us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:44.239908 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:44.239925 (+ 17us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:44.240007 (+ 82us) tablet.cc:701] Row locks acquired
0618 13:59:44.240009 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:44.240046 (+ 37us) write_op.cc:270] Start()
0618 13:59:44.240094 (+ 48us) write_op.cc:276] Timestamp: P: 1781791184239994 usec, L: 0
0618 13:59:44.240097 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:44.240296 (+ 199us) log.cc:844] Serialized 356 byte log entry
0618 13:59:44.243067 (+ 2771us) op_driver.cc:464] REPLICATION: finished
0618 13:59:44.244441 (+ 1374us) write_op.cc:301] APPLY: starting
0618 13:59:44.244457 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:44.244499 (+ 42us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:44.244545 (+ 46us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:44.244599 (+ 54us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:44.244633 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:44.244639 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:44.244733 (+ 94us) log.cc:844] Serialized 23 byte log entry
0618 13:59:44.244956 (+ 223us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.244975 (+ 19us) write_op.cc:454] Released schema lock
0618 13:59:44.244982 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0618 13:59:44.704339 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:44.704460 (+ 121us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:44.704467 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:44.704471 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:44.704565 (+ 94us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:44.704577 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:44.704584 (+ 7us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:44.704644 (+ 60us) tablet.cc:701] Row locks acquired
0618 13:59:44.704648 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:44.704680 (+ 32us) write_op.cc:270] Start()
0618 13:59:44.704711 (+ 31us) write_op.cc:276] Timestamp: P: 1781791184704635 usec, L: 0
0618 13:59:44.704716 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:44.704874 (+ 158us) log.cc:844] Serialized 356 byte log entry
0618 13:59:44.711794 (+ 6920us) op_driver.cc:464] REPLICATION: finished
0618 13:59:44.716131 (+ 4337us) write_op.cc:301] APPLY: starting
0618 13:59:44.716141 (+ 10us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:44.716172 (+ 31us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:44.716194 (+ 22us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:44.716230 (+ 36us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:44.716253 (+ 23us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:44.716259 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:44.716310 (+ 51us) log.cc:844] Serialized 23 byte log entry
0618 13:59:44.719265 (+ 2955us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.719284 (+ 19us) write_op.cc:454] Released schema lock
0618 13:59:44.719290 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0618 13:59:44.733248 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:44.733348 (+ 100us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:44.733356 (+ 8us) write_op.cc:435] Acquired schema lock
0618 13:59:44.733359 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:44.733466 (+ 107us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:44.733482 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:44.733488 (+ 6us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:44.733552 (+ 64us) tablet.cc:701] Row locks acquired
0618 13:59:44.733554 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:44.733593 (+ 39us) write_op.cc:270] Start()
0618 13:59:44.733630 (+ 37us) write_op.cc:276] Timestamp: P: 1781791184733544 usec, L: 0
0618 13:59:44.733633 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:44.736336 (+ 2703us) log.cc:844] Serialized 356 byte log entry
0618 13:59:44.859506 (+123170us) op_driver.cc:464] REPLICATION: finished
0618 13:59:44.859663 (+ 157us) write_op.cc:301] APPLY: starting
0618 13:59:44.859691 (+ 28us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:44.859749 (+ 58us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:44.859787 (+ 38us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:44.859857 (+ 70us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:44.859909 (+ 52us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:44.859920 (+ 11us) write_op.cc:312] APPLY: finished
0618 13:59:44.860070 (+ 150us) log.cc:844] Serialized 23 byte log entry
0618 13:59:44.860132 (+ 62us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.860158 (+ 26us) write_op.cc:454] Released schema lock
0618 13:59:44.860167 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:44.860256 (+ 89us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.860263 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":86429,"reported_tablets":159,"child_traces":[["op",{"apply.queue_time_us":1304,"apply.run_cpu_time_us":443,"apply.run_wall_time_us":599,"num_ops":1,"prepare.queue_time_us":12294,"prepare.run_cpu_time_us":805,"prepare.run_wall_time_us":806,"raft.queue_time_us":199,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":244,"replication_time_us":2941}],["op",{"apply.queue_time_us":4303,"apply.run_cpu_time_us":337,"apply.run_wall_time_us":3222,"num_ops":1,"prepare.queue_time_us":12262,"prepare.run_cpu_time_us":651,"prepare.run_wall_time_us":4561,"raft.queue_time_us":1088,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":7055,"thread_start_us":16228,"threads_started":1}],["op",{"apply.queue_time_us":43,"apply.run_cpu_time_us":593,"apply.run_wall_time_us":594,"num_ops":1,"prepare.queue_time_us":3868,"prepare.run_cpu_time_us":801,"prepare.run_wall_time_us":3293,"raft.queue_time_us":45,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":125837}]]}
I20260618 13:59:45.031769 29024 master_service.cc:960] ReplaceTablet: received request to replace tablet 91d014b46a0f448b929cbc9287ead0a7 from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:45.040316 28797 tablet_bootstrap.cc:654] T 779755a607bf46219900412b527e58b5 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.040465 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 310) took 3757 ms. Trace:
I20260618 13:59:45.040556 29566 rpcz_store.cc:276] 0618 13:59:41.282605 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.282658 (+ 53us) service_pool.cc:224] Handling call
0618 13:59:45.040354 (+3757696us) ts_tablet_manager.cc:1151] Deleting tablet 37a6764a5e024903848561c9935b1a43
0618 13:59:45.040366 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.040447 (+ 81us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3716027}
W20260618 13:59:45.041030 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 37a6764a5e024903848561c9935b1a43 because the tablet was not found. No further retry: Not found: Tablet not found: 37a6764a5e024903848561c9935b1a43
I20260618 13:59:45.040982 30262 raft_consensus.cc:493] T 7c40fe72e00a447582a35611de8ffa7c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.041177 30262 raft_consensus.cc:515] T 7c40fe72e00a447582a35611de8ffa7c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.041865 30262 leader_election.cc:290] T 7c40fe72e00a447582a35611de8ffa7c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:44.964716 29011 rpcz_store.cc:269] 0618 13:59:41.165470 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.166806 (+ 1336us) service_pool.cc:224] Handling call
0618 13:59:44.964490 (+3797684us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:42.670533 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:42.670668 (+ 135us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:42.670675 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:42.670678 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:42.670845 (+ 167us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:42.670860 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:42.670865 (+ 5us) tablet.cc:685] Acquiring locks for 4 operations
0618 13:59:42.670970 (+ 105us) tablet.cc:701] Row locks acquired
0618 13:59:42.670973 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:42.671010 (+ 37us) write_op.cc:270] Start()
0618 13:59:42.671054 (+ 44us) write_op.cc:276] Timestamp: P: 1781791182670966 usec, L: 0
0618 13:59:42.671058 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:42.671278 (+ 220us) log.cc:844] Serialized 1662 byte log entry
0618 13:59:42.676454 (+ 5176us) op_driver.cc:464] REPLICATION: finished
0618 13:59:42.678789 (+ 2335us) write_op.cc:301] APPLY: starting
0618 13:59:42.678804 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:42.678847 (+ 43us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:42.678874 (+ 27us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:42.678951 (+ 77us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:42.678981 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0618 13:59:42.678988 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:42.679073 (+ 85us) log.cc:844] Serialized 41 byte log entry
0618 13:59:42.679139 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:42.679161 (+ 22us) write_op.cc:454] Released schema lock
0618 13:59:42.679167 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:42.679250 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:42.679254 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:43.041281 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:43.041384 (+ 103us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:43.041391 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:43.041394 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:43.041487 (+ 93us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:43.041496 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:43.041500 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:43.041549 (+ 49us) tablet.cc:701] Row locks acquired
0618 13:59:43.041551 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:43.041581 (+ 30us) write_op.cc:270] Start()
0618 13:59:43.041608 (+ 27us) write_op.cc:276] Timestamp: P: 1781791183041554 usec, L: 0
0618 13:59:43.041612 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:43.041759 (+ 147us) log.cc:844] Serialized 356 byte log entry
0618 13:59:43.081235 (+ 39476us) op_driver.cc:464] REPLICATION: finished
0618 13:59:43.082162 (+ 927us) write_op.cc:301] APPLY: starting
0618 13:59:43.082179 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:43.082221 (+ 42us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:43.082263 (+ 42us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:43.082315 (+ 52us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:43.082351 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:43.082359 (+ 8us) write_op.cc:312] APPLY: finished
0618 13:59:43.082465 (+ 106us) log.cc:844] Serialized 23 byte log entry
0618 13:59:43.082537 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:43.082556 (+ 19us) write_op.cc:454] Released schema lock
0618 13:59:43.082565 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:43.082650 (+ 85us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:43.082656 (+ 6us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:43.122116 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:43.122249 (+ 133us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:43.122256 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:43.122259 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:43.122365 (+ 106us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:43.122376 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:43.122380 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:43.122476 (+ 96us) tablet.cc:701] Row locks acquired
0618 13:59:43.122479 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:43.122525 (+ 46us) write_op.cc:270] Start()
0618 13:59:43.122558 (+ 33us) write_op.cc:276] Timestamp: P: 1781791183122489 usec, L: 0
0618 13:59:43.122563 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:43.122725 (+ 162us) log.cc:844] Serialized 356 byte log entry
0618 13:59:43.172847 (+ 50122us) op_driver.cc:464] REPLICATION: finished
0618 13:59:43.173570 (+ 723us) write_op.cc:301] APPLY: starting
0618 13:59:43.173581 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:43.173606 (+ 25us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:43.173637 (+ 31us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:43.173670 (+ 33us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:43.173693 (+ 23us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:43.173699 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:43.173751 (+ 52us) log.cc:844] Serialized 23 byte log entry
0618 13:59:43.173785 (+ 34us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:43.173795 (+ 10us) write_op.cc:454] Released schema lock
0618 13:59:43.173800 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:43.173853 (+ 53us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:43.173856 (+ 3us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:43.846129 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:43.846244 (+ 115us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:43.846250 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:43.846252 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:43.846354 (+ 102us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:43.846364 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:43.846368 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:43.846742 (+ 374us) tablet.cc:701] Row locks acquired
0618 13:59:43.846745 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:43.846779 (+ 34us) write_op.cc:270] Start()
0618 13:59:43.846813 (+ 34us) write_op.cc:276] Timestamp: P: 1781791183846705 usec, L: 0
0618 13:59:43.846816 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:43.846977 (+ 161us) log.cc:844] Serialized 356 byte log entry
0618 13:59:44.025258 (+178281us) op_driver.cc:464] REPLICATION: finished
0618 13:59:44.025496 (+ 238us) write_op.cc:301] APPLY: starting
0618 13:59:44.025515 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:44.025565 (+ 50us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:44.025592 (+ 27us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:44.025647 (+ 55us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:44.025683 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:44.025690 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:44.025769 (+ 79us) log.cc:844] Serialized 23 byte log entry
0618 13:59:44.027804 (+ 2035us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.027824 (+ 20us) write_op.cc:454] Released schema lock
0618 13:59:44.027831 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:44.027982 (+ 151us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.027987 (+ 5us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:44.678696 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:44.678798 (+ 102us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:44.678803 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:44.678805 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:44.678890 (+ 85us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:44.678900 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:44.678905 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:44.678958 (+ 53us) tablet.cc:701] Row locks acquired
0618 13:59:44.678961 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:44.678991 (+ 30us) write_op.cc:270] Start()
0618 13:59:44.679017 (+ 26us) write_op.cc:276] Timestamp: P: 1781791184678950 usec, L: 0
0618 13:59:44.679020 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:44.679154 (+ 134us) log.cc:844] Serialized 356 byte log entry
0618 13:59:44.682969 (+ 3815us) op_driver.cc:464] REPLICATION: finished
0618 13:59:44.683125 (+ 156us) write_op.cc:301] APPLY: starting
0618 13:59:44.683142 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:44.683185 (+ 43us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:44.683225 (+ 40us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:44.683278 (+ 53us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:44.683316 (+ 38us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:44.683326 (+ 10us) write_op.cc:312] APPLY: finished
0618 13:59:44.683401 (+ 75us) log.cc:844] Serialized 23 byte log entry
0618 13:59:44.683459 (+ 58us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.683481 (+ 22us) write_op.cc:454] Released schema lock
0618 13:59:44.683488 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:44.683578 (+ 90us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.683583 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":105281,"reported_tablets":140,"child_traces":[["op",{"apply.queue_time_us":2279,"apply.run_cpu_time_us":456,"apply.run_wall_time_us":455,"mutex_wait_us":19,"num_ops":4,"prepare.queue_time_us":3503,"prepare.run_cpu_time_us":909,"prepare.run_wall_time_us":919,"raft.queue_time_us":145,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":283,"replication_time_us":5365}],["op",{"apply.queue_time_us":858,"apply.run_cpu_time_us":480,"apply.run_wall_time_us":480,"mutex_wait_us":4153,"num_ops":1,"prepare.queue_time_us":4201,"prepare.run_cpu_time_us":566,"prepare.run_wall_time_us":566,"raft.queue_time_us":657,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":39595}],["op",{"apply.queue_time_us":690,"apply.run_cpu_time_us":276,"apply.run_wall_time_us":276,"num_ops":1,"prepare.queue_time_us":37262,"prepare.run_cpu_time_us":690,"prepare.run_wall_time_us":688,"raft.queue_time_us":90,"raft.run_cpu_time_us":2,"raft.run_wall_time_us":3,"replication_time_us":50263}],["op",{"apply.queue_time_us":182,"apply.run_cpu_time_us":456,"apply.run_wall_time_us":2475,"num_ops":1,"prepare.queue_time_us":53386,"prepare.run_cpu_time_us":683,"prepare.run_wall_time_us":1323,"raft.queue_time_us":36,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":178410}],["op",{"apply.queue_time_us":103,"apply.run_cpu_time_us":442,"apply.run_wall_time_us":441,"num_ops":1,"prepare.queue_time_us":42,"prepare.run_cpu_time_us":563,"prepare.run_wall_time_us":563,"raft.queue_time_us":183,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":275,"replication_time_us":3917}]]}
I20260618 13:59:45.045253 29011 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_1aaf686e997e41f490f09c2e23259f74"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:45.045989 29011 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1aaf686e997e41f490f09c2e23259f74 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:45.046937 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 311) took 3764 ms. Trace:
I20260618 13:59:45.047072 29566 rpcz_store.cc:276] 0618 13:59:41.282737 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.282782 (+ 45us) service_pool.cc:224] Handling call
0618 13:59:45.046847 (+3764065us) ts_tablet_manager.cc:1151] Deleting tablet dd2385e6752d49429422fc854539e9c6
0618 13:59:45.046857 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.046919 (+ 62us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3722465}
I20260618 13:59:45.047583 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 312) took 3764 ms. Trace:
W20260618 13:59:45.047578 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet dd2385e6752d49429422fc854539e9c6 because the tablet was not found. No further retry: Not found: Tablet not found: dd2385e6752d49429422fc854539e9c6
I20260618 13:59:45.047699 29566 rpcz_store.cc:276] 0618 13:59:41.282842 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.282888 (+ 46us) service_pool.cc:224] Handling call
0618 13:59:45.047496 (+3764608us) ts_tablet_manager.cc:1151] Deleting tablet f89c9954d48e407c96c294c26644e25c
0618 13:59:45.047516 (+ 20us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.047571 (+ 55us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3723071}
I20260618 13:59:45.048132 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 314) took 3765 ms. Trace:
I20260618 13:59:45.048199 29566 rpcz_store.cc:276] 0618 13:59:41.283073 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.283119 (+ 46us) service_pool.cc:224] Handling call
0618 13:59:45.048054 (+3764935us) ts_tablet_manager.cc:1151] Deleting tablet 36b8a46a41e543c58a946b6764618c7d
0618 13:59:45.048064 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.048119 (+ 55us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3723521}
I20260618 13:59:45.048527 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 306) took 3766 ms. Trace:
I20260618 13:59:45.048593 29566 rpcz_store.cc:276] 0618 13:59:41.282018 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.283159 (+ 1141us) service_pool.cc:224] Handling call
0618 13:59:45.048459 (+3765300us) ts_tablet_manager.cc:1151] Deleting tablet 4d6b01b897134d5bbd6a7d93b82ddb5e
0618 13:59:45.048467 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.048516 (+ 49us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3723883}
I20260618 13:59:45.048902 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 307) took 3766 ms. Trace:
I20260618 13:59:45.048960 29566 rpcz_store.cc:276] 0618 13:59:41.282188 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.283305 (+ 1117us) service_pool.cc:224] Handling call
0618 13:59:45.048847 (+3765542us) ts_tablet_manager.cc:1151] Deleting tablet d1baf86497d442dbbbddbc3498bebbee
0618 13:59:45.048854 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.048892 (+ 38us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3724230}
I20260618 13:59:45.049256 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 308) took 3766 ms. Trace:
I20260618 13:59:45.049309 29566 rpcz_store.cc:276] 0618 13:59:41.282295 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.283421 (+ 1126us) service_pool.cc:224] Handling call
0618 13:59:45.049202 (+3765781us) ts_tablet_manager.cc:1151] Deleting tablet 7aa9f65d8a5c46fdb745ee17af946dce
0618 13:59:45.049209 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.049247 (+ 38us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3724548}
I20260618 13:59:45.049626 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 313) took 3766 ms. Trace:
I20260618 13:59:45.049682 29566 rpcz_store.cc:276] 0618 13:59:41.282949 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.283533 (+ 584us) service_pool.cc:224] Handling call
0618 13:59:45.049568 (+3766035us) ts_tablet_manager.cc:1151] Deleting tablet e95c88e4b0a046a3ad69ee49af47c7c3
0618 13:59:45.049575 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.049615 (+ 40us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3724873}
I20260618 13:59:45.049997 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 315) took 3765 ms. Trace:
I20260618 13:59:45.050050 29566 rpcz_store.cc:276] 0618 13:59:41.284659 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.285193 (+ 534us) service_pool.cc:224] Handling call
0618 13:59:45.049942 (+3764749us) ts_tablet_manager.cc:1151] Deleting tablet d502d67826ff4d7a8b0f515e5d8b6d77
0618 13:59:45.049949 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.049988 (+ 39us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3725065}
I20260618 13:59:45.050350 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 316) took 3765 ms. Trace:
W20260618 13:59:45.048766 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet f89c9954d48e407c96c294c26644e25c because the tablet was not found. No further retry: Not found: Tablet not found: f89c9954d48e407c96c294c26644e25c
W20260618 13:59:45.059132 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 36b8a46a41e543c58a946b6764618c7d because the tablet was not found. No further retry: Not found: Tablet not found: 36b8a46a41e543c58a946b6764618c7d
W20260618 13:59:45.059437 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 4d6b01b897134d5bbd6a7d93b82ddb5e because the tablet was not found. No further retry: Not found: Tablet not found: 4d6b01b897134d5bbd6a7d93b82ddb5e
W20260618 13:59:45.060096 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d1baf86497d442dbbbddbc3498bebbee because the tablet was not found. No further retry: Not found: Tablet not found: d1baf86497d442dbbbddbc3498bebbee
W20260618 13:59:45.060477 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 7aa9f65d8a5c46fdb745ee17af946dce because the tablet was not found. No further retry: Not found: Tablet not found: 7aa9f65d8a5c46fdb745ee17af946dce
W20260618 13:59:45.060731 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet e95c88e4b0a046a3ad69ee49af47c7c3 because the tablet was not found. No further retry: Not found: Tablet not found: e95c88e4b0a046a3ad69ee49af47c7c3
W20260618 13:59:45.061393 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d502d67826ff4d7a8b0f515e5d8b6d77 because the tablet was not found. No further retry: Not found: Tablet not found: d502d67826ff4d7a8b0f515e5d8b6d77
I20260618 13:59:45.062983 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e25027f423144b088dad955c7282921 with delete_type TABLET_DATA_DELETED (replaced by tablet a8da96fc621c47d4a444b5a39259f85d) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.063544 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 55ae29c3b7ed44cfa659130745037aa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:31 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.063961 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet c3d523419ff244f085d6eac1a3231973 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.064353 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet f3ba9186b3ff47d5a2b158fef619686e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.064625 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet c4cfca9e76e641a7bc0b58b9ee240f36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:30 UTC) from {username='slave'} at 127.0.0.1:57282
W20260618 13:59:45.053333 29025 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.2:52853 (request call id 126) took 2392 ms (client timeout 1000 ms). Trace:
W20260618 13:59:45.064920 29025 rpcz_store.cc:269] 0618 13:59:42.660845 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:43.031510 (+370665us) service_pool.cc:224] Handling call
0618 13:59:45.053299 (+2021789us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":144650,"reported_tablets":157}
I20260618 13:59:45.067005 29025 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_73e7e7a1c4ad48d59b10333525a7d129"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:45.068734 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet 35d8dd2c388e43a0be380aa953ec2c82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:34 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.069692 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 7aa9f65d8a5c46fdb745ee17af946dce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.070364 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet 18d8b609b96a4e81b57f69bff522eeea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.006145 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c04f55890648f0a8184f3783243f7c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:45.070611 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f3d2f236fd44ab8af0524ea282cfa94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.072095 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet c7f42997416841a7b2c1ad347d2e9282 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.072763 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 125bb6885ab348d885d5a3b4bdd45bc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:21 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.073115 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet c552d1316e3340f4bcfa39618ae309b8 with delete_type TABLET_DATA_DELETED (replaced by tablet fe750fc5d998448caeab0a6c808dfba7) from {username='slave'} at 127.0.0.1:57282
W20260618 13:59:45.073601 29025 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_73e7e7a1c4ad48d59b10333525a7d129 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:45.073782 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 7d4144b1b5ea4e968672616bc22bead8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.074108 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 827cc0987bd541759e334fb620f04ed5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:25 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.074980 30950 tablet_replica.cc:333] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:45.054462 30854 raft_consensus.cc:493] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.050405 29566 rpcz_store.cc:276] 0618 13:59:41.284852 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.285345 (+ 493us) service_pool.cc:224] Handling call
0618 13:59:45.050295 (+3764950us) ts_tablet_manager.cc:1151] Deleting tablet bce7bf74b6d24d0bb43808b8a62cf1b7
0618 13:59:45.050301 (+ 6us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.050340 (+ 39us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3725366}
I20260618 13:59:45.075104 30854 raft_consensus.cc:515] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.075178 30950 raft_consensus.cc:2243] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:45.075310 30950 raft_consensus.cc:2272] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:45.075430 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet bce7bf74b6d24d0bb43808b8a62cf1b7 because the tablet was not found. No further retry: Not found: Tablet not found: bce7bf74b6d24d0bb43808b8a62cf1b7
I20260618 13:59:45.075672 30854 leader_election.cc:290] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.075817 30950 ts_tablet_manager.cc:1916] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:45.070912 28618 leader_election.cc:343] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.076570 28618 leader_election.cc:304] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.076532 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b44c3b0c152442a78edc4736da2cb15c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:45.076705 28402 raft_consensus.cc:2468] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:45.077107 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 317) took 3792 ms. Trace:
I20260618 13:59:45.077198 29566 rpcz_store.cc:276] 0618 13:59:41.284944 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.285486 (+ 542us) service_pool.cc:224] Handling call
0618 13:59:45.077007 (+3791521us) ts_tablet_manager.cc:1151] Deleting tablet b0285c75c632457383e81805ab036aaf
0618 13:59:45.077019 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.077088 (+ 69us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3752027}
W20260618 13:59:45.077669 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet b0285c75c632457383e81805ab036aaf because the tablet was not found. No further retry: Not found: Tablet not found: b0285c75c632457383e81805ab036aaf
I20260618 13:59:45.078008 30890 raft_consensus.cc:2749] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:45.068326 30847 raft_consensus.cc:493] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.079466 30847 raft_consensus.cc:515] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.080382 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89b26d1595642ff82381f8baddfb99a" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:45.080483 29017 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.1:48495 (request call id 154) took 1342 ms (client timeout 1000 ms). Trace:
W20260618 13:59:45.080806 28618 leader_election.cc:343] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:45.080600 29017 rpcz_store.cc:269] 0618 13:59:43.737887 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:43.826255 (+ 88368us) service_pool.cc:224] Handling call
0618 13:59:45.080450 (+1254195us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:44.832242 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:44.832378 (+ 136us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:44.832385 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:44.832387 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:44.832539 (+ 152us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:44.832551 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:44.832556 (+ 5us) tablet.cc:685] Acquiring locks for 3 operations
0618 13:59:44.832647 (+ 91us) tablet.cc:701] Row locks acquired
0618 13:59:44.832649 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:44.832683 (+ 34us) write_op.cc:270] Start()
0618 13:59:44.832718 (+ 35us) write_op.cc:276] Timestamp: P: 1781791184832626 usec, L: 0
0618 13:59:44.832721 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:44.832907 (+ 186us) log.cc:844] Serialized 1284 byte log entry
0618 13:59:44.859587 (+ 26680us) op_driver.cc:464] REPLICATION: finished
0618 13:59:44.859719 (+ 132us) write_op.cc:301] APPLY: starting
0618 13:59:44.859732 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:44.859772 (+ 40us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:44.859803 (+ 31us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:44.859894 (+ 91us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:44.859933 (+ 39us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0618 13:59:44.859940 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:44.860033 (+ 93us) log.cc:844] Serialized 35 byte log entry
0618 13:59:44.860092 (+ 59us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.860111 (+ 19us) write_op.cc:454] Released schema lock
0618 13:59:44.860117 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:44.860184 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.860187 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":118006,"reported_tablets":145,"child_traces":[["op",{"apply.queue_time_us":98,"apply.run_cpu_time_us":457,"apply.run_wall_time_us":457,"num_ops":3,"prepare.queue_time_us":25682,"prepare.run_cpu_time_us":794,"prepare.run_wall_time_us":794,"raft.queue_time_us":62,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":4,"replication_time_us":26849}]]}
I20260618 13:59:45.081141 30847 leader_election.cc:290] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.081429 28617 leader_election.cc:304] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:45.081498 29017 master_service.cc:960] ReplaceTablet: received request to replace tablet 962bf304a285412fb2a529db5b994e9e from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:45.081846 30847 raft_consensus.cc:2804] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:45.081919 30847 raft_consensus.cc:493] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:45.081887 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89b26d1595642ff82381f8baddfb99a" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:45.081979 30847 raft_consensus.cc:3060] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.082052 28402 raft_consensus.cc:2468] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:45.084506 28617 leader_election.cc:304] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.084867 30854 raft_consensus.cc:2804] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:45.084939 30854 raft_consensus.cc:493] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:45.085005 30854 raft_consensus.cc:3060] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.079586 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 318) took 3794 ms. Trace:
I20260618 13:59:45.085104 29566 rpcz_store.cc:276] 0618 13:59:41.285029 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.285641 (+ 612us) service_pool.cc:224] Handling call
0618 13:59:45.079259 (+3793618us) ts_tablet_manager.cc:1151] Deleting tablet fe750fc5d998448caeab0a6c808dfba7
0618 13:59:45.079267 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.079566 (+ 299us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3754245}
W20260618 13:59:45.085603 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet fe750fc5d998448caeab0a6c808dfba7 because the tablet was not found. No further retry: Not found: Tablet not found: fe750fc5d998448caeab0a6c808dfba7
I20260618 13:59:45.086366 29566 tablet_replica.cc:333] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:45.086568 29566 raft_consensus.cc:2243] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:45.086696 29566 raft_consensus.cc:2272] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.087699 29566 ts_tablet_manager.cc:1916] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.087888 30854 raft_consensus.cc:515] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.088547 30854 leader_election.cc:290] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.089744 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89b26d1595642ff82381f8baddfb99a" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:45.090468 28618 leader_election.cc:343] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.091223 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 864f2602840c40fc8606b3784e671cba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:23 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.091202 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89b26d1595642ff82381f8baddfb99a" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:45.091361 28402 raft_consensus.cc:3060] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.091676 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 591465e2b57247a2b224cc8b09225811 with delete_type TABLET_DATA_DELETED (replaced by tablet d4667fe48f0f4eff9c6c8baabdab58d4) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.091975 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet cd06bb7cffc34412b420d70435fce373 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.092268 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 with delete_type TABLET_DATA_DELETED (replaced by tablet cea4bc13eabc4dd98b4f96723498014c) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.092602 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet e6de1293572f4e75af3417f281b9a142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.092888 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet f278d0acbb0c4a0ca61640bc142e6335 with delete_type TABLET_DATA_DELETED (replaced by tablet 72c04f55890648f0a8184f3783243f7c) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.093202 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 02b5cc08ca0c4e719e0414b345211ba5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:26 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.093526 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet cea4bc13eabc4dd98b4f96723498014c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:57282
W20260618 13:59:45.093894 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet c5d046e0421142cd84c5b12d6a434f53 because the tablet was not found. No further retry: Not found: Tablet not found: c5d046e0421142cd84c5b12d6a434f53
I20260618 13:59:45.094003 28402 raft_consensus.cc:2468] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:45.094373 28617 leader_election.cc:304] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.094735 30856 raft_consensus.cc:2804] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:45.094853 30856 raft_consensus.cc:697] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:45.095117 30856 consensus_queue.cc:237] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.095477 28645 tablet_service.cc:1558] Processing DeleteTablet for tablet ba2d6894bf5649fc8cfae7e033c432b9 with delete_type TABLET_DATA_DELETED (replaced by tablet a6ba4976bb6e4c2da3a563ccad4eb2b0) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:45.096827 29019 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.3:42065 (request call id 150) took 2897 ms (client timeout 1000 ms). Trace:
I20260618 13:59:45.096187 28797 tablet_bootstrap.cc:492] T 779755a607bf46219900412b527e58b5 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
W20260618 13:59:45.097290 29019 rpcz_store.cc:269] 0618 13:59:42.199408 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:42.641578 (+442170us) service_pool.cc:224] Handling call
0618 13:59:45.096760 (+2455182us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:43.039517 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:43.039648 (+ 131us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:43.039654 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:43.039656 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:43.039903 (+ 247us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:43.039918 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:43.039922 (+ 4us) tablet.cc:685] Acquiring locks for 6 operations
0618 13:59:43.040051 (+ 129us) tablet.cc:701] Row locks acquired
0618 13:59:43.040054 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:43.040093 (+ 39us) write_op.cc:270] Start()
0618 13:59:43.040128 (+ 35us) write_op.cc:276] Timestamp: P: 1781791183040065 usec, L: 0
0618 13:59:43.040132 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:43.040363 (+ 231us) log.cc:844] Serialized 2416 byte log entry
0618 13:59:43.067939 (+ 27576us) op_driver.cc:464] REPLICATION: finished
0618 13:59:43.068109 (+ 170us) write_op.cc:301] APPLY: starting
0618 13:59:43.068128 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:43.068186 (+ 58us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:43.068215 (+ 29us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:43.068341 (+ 126us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:43.068378 (+ 37us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0618 13:59:43.068385 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:43.068492 (+ 107us) log.cc:844] Serialized 53 byte log entry
0618 13:59:43.068568 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:43.068589 (+ 21us) write_op.cc:454] Released schema lock
0618 13:59:43.068596 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:43.068666 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:43.068669 (+ 3us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:43.847486 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:43.847590 (+ 104us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:43.847597 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:43.847601 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:43.847693 (+ 92us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:43.847702 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:43.847708 (+ 6us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:43.847779 (+ 71us) tablet.cc:701] Row locks acquired
0618 13:59:43.847784 (+ 5us) write_op.cc:260] PREPARE: finished
0618 13:59:43.847819 (+ 35us) write_op.cc:270] Start()
0618 13:59:43.847867 (+ 48us) write_op.cc:276] Timestamp: P: 1781791183847742 usec, L: 0
0618 13:59:43.847872 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:43.848052 (+ 180us) log.cc:844] Serialized 356 byte log entry
0618 13:59:44.025330 (+177278us) op_driver.cc:464] REPLICATION: finished
0618 13:59:44.026236 (+ 906us) write_op.cc:301] APPLY: starting
0618 13:59:44.026250 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:44.026282 (+ 32us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:44.026323 (+ 41us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:44.026379 (+ 56us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:44.026409 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:44.026439 (+ 30us) write_op.cc:312] APPLY: finished
0618 13:59:44.026521 (+ 82us) log.cc:844] Serialized 23 byte log entry
0618 13:59:44.028400 (+ 1879us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.028412 (+ 12us) write_op.cc:454] Released schema lock
0618 13:59:44.028416 (+ 4us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:44.043822 (+ 15406us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.043827 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":15764,"reported_tablets":165,"child_traces":[["op",{"apply.queue_time_us":115,"apply.run_cpu_time_us":550,"apply.run_wall_time_us":549,"num_ops":6,"prepare.queue_time_us":8185,"prepare.run_cpu_time_us":984,"prepare.run_wall_time_us":983,"raft.queue_time_us":921,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":246,"replication_time_us":27768}],["op",{"apply.queue_time_us":850,"apply.run_cpu_time_us":417,"apply.run_wall_time_us":17581,"num_ops":1,"prepare.queue_time_us":43098,"prepare.run_cpu_time_us":725,"prepare.run_wall_time_us":39283,"raft.queue_time_us":51,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":6,"replication_time_us":177440,"thread_start_us":1354,"threads_started":1}]]}
I20260618 13:59:45.097858 28797 ts_tablet_manager.cc:1403] T 779755a607bf46219900412b527e58b5 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.069s user 0.009s sys 0.009s
I20260618 13:59:45.098829 29019 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_1b1680a2f241430aa5d5702997655d36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:45.103253 28797 raft_consensus.cc:359] T 779755a607bf46219900412b527e58b5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.106117 28797 raft_consensus.cc:385] T 779755a607bf46219900412b527e58b5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.106554 28797 raft_consensus.cc:740] T 779755a607bf46219900412b527e58b5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.108893 28798 tablet_bootstrap.cc:492] T e3a3395aa90e426eb48a3920a8c45ba8 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:45.109035 28798 ts_tablet_manager.cc:1403] T e3a3395aa90e426eb48a3920a8c45ba8 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.100s user 0.013s sys 0.020s
I20260618 13:59:45.109211 28797 consensus_queue.cc:260] T 779755a607bf46219900412b527e58b5 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.109783 28798 raft_consensus.cc:359] T e3a3395aa90e426eb48a3920a8c45ba8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.109939 28798 raft_consensus.cc:385] T e3a3395aa90e426eb48a3920a8c45ba8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.109992 28798 raft_consensus.cc:740] T e3a3395aa90e426eb48a3920a8c45ba8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.110181 28798 consensus_queue.cc:260] T e3a3395aa90e426eb48a3920a8c45ba8 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.111183 28798 ts_tablet_manager.cc:1434] T e3a3395aa90e426eb48a3920a8c45ba8 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:45.112149 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 6eea4d59935d40aa97dc10ec7469aa56 with delete_type TABLET_DATA_DELETED (replaced by tablet 4bc87770baa34842a25ed97a19354446) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.112449 28798 tablet_bootstrap.cc:492] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:45.112614 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 0463fa2e30554abcbfc8f37b702e81d4 with delete_type TABLET_DATA_DELETED (replaced by tablet fe1139c409e84070a2f496a85545e8d5) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.112951 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet be59f988f0e34b9480f859cf88b9c223 with delete_type TABLET_DATA_DELETED (replaced by tablet b968a8d4014341ca9c5964263721ac79) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.110317 30847 raft_consensus.cc:515] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.114280 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b44c3b0c152442a78edc4736da2cb15c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:45.114363 30847 leader_election.cc:290] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.114464 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b44c3b0c152442a78edc4736da2cb15c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:45.114641 28402 raft_consensus.cc:3060] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.114758 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b44c3b0c152442a78edc4736da2cb15c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:45.114785 28618 leader_election.cc:343] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:45.115176 28618 leader_election.cc:343] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.115453 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet ca2ee746c5024cfc9b31c470f9616e7d with delete_type TABLET_DATA_DELETED (replaced by tablet 60b6216474b74a14af6abf977e312b77) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.115713 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet 68b8293a7c924a449307ab88a1cf9290 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:24 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.113279 28797 ts_tablet_manager.cc:1434] T 779755a607bf46219900412b527e58b5 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.015s user 0.001s sys 0.001s
I20260618 13:59:45.116541 28797 tablet_bootstrap.cc:492] T 5ffd607dd9eb43e0a3769fcdd28ed05f P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:45.117879 28797 tablet_bootstrap.cc:654] T 5ffd607dd9eb43e0a3769fcdd28ed05f P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.117856 28402 raft_consensus.cc:2468] T b44c3b0c152442a78edc4736da2cb15c P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:45.118384 28617 leader_election.cc:304] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.118798 30854 raft_consensus.cc:2804] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:45.119076 30854 raft_consensus.cc:697] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:45.119359 30854 consensus_queue.cc:237] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.120774 30152 raft_consensus.cc:2749] T b89b26d1595642ff82381f8baddfb99a P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:45.121773 28798 tablet_bootstrap.cc:654] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:45.121477 28346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 154034us
W20260618 13:59:45.122538 28346 leader_election.cc:343] T 134a43259d5b429fbe47d0555d9942f3 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.122984 28346 leader_election.cc:304] T 134a43259d5b429fbe47d0555d9942f3 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:45.123626 30152 raft_consensus.cc:2749] T 134a43259d5b429fbe47d0555d9942f3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:45.131927 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 8fea86ca38a741238a59053151087f69 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f9ec5f31f6c4d37b7c3ec585ed3211b) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:45.132321 29566 ts_tablet_manager.cc:1929] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:45.133792 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet fea86899d2c047d997cebe80551e83c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:38230
W20260618 13:59:45.133810 29019 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1b1680a2f241430aa5d5702997655d36 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:45.134198 29566 log.cc:1199] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/72c04f55890648f0a8184f3783243f7c
W20260618 13:59:45.112226 29007 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.2:52853 (request call id 124) took 3988 ms (client timeout 1000 ms). Trace:
I20260618 13:59:45.130406 29011 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 139) took 3479 ms. Trace:
I20260618 13:59:45.135003 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 8ac84641d0f1422193e42a42468789e1 (DEFAULT_TABLE table=default.Table_e98f724cea1e4b7c8acd2a357fb845a0 [id=95cb33ea8eef404e87348fda4b2f1805]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.135397 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ac84641d0f1422193e42a42468789e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.135025 29011 rpcz_store.cc:276] 0618 13:59:41.650719 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:45.044998 (+3394279us) service_pool.cc:224] Handling call
0618 13:59:45.057951 (+ 12953us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:45.061392 (+ 3441us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:45.071561 (+ 10169us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.071570 (+ 9us) write_op.cc:454] Released schema lock
0618 13:59:45.075621 (+ 4051us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:45.075749 (+ 128us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:45.130331 (+ 54582us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:45.062761 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:45.062894 (+ 133us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:45.062900 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:45.062902 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:45.062990 (+ 88us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:45.063003 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:45.063007 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:45.063097 (+ 90us) tablet.cc:701] Row locks acquired
0618 13:59:45.063100 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:45.063144 (+ 44us) write_op.cc:270] Start()
0618 13:59:45.063187 (+ 43us) write_op.cc:276] Timestamp: P: 1781791185063113 usec, L: 0
0618 13:59:45.063193 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:45.063416 (+ 223us) log.cc:844] Serialized 433 byte log entry
0618 13:59:45.066091 (+ 2675us) op_driver.cc:464] REPLICATION: finished
0618 13:59:45.066216 (+ 125us) write_op.cc:301] APPLY: starting
0618 13:59:45.066230 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:45.066273 (+ 43us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:45.066297 (+ 24us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:45.066453 (+ 156us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:45.066487 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:45.066494 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:45.066576 (+ 82us) log.cc:844] Serialized 29 byte log entry
0618 13:59:45.066634 (+ 58us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.066652 (+ 18us) write_op.cc:454] Released schema lock
0618 13:59:45.066658 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":83,"apply.run_cpu_time_us":488,"apply.run_wall_time_us":487,"num_ops":2,"prepare.queue_time_us":49,"prepare.run_cpu_time_us":788,"prepare.run_wall_time_us":789,"raft.queue_time_us":806,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":205,"replication_time_us":2868}]]}
I20260618 13:59:45.135809 29011 master_service.cc:960] ReplaceTablet: received request to replace tablet c4cfca9e76e641a7bc0b58b9ee240f36 from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:45.136026 28798 tablet_bootstrap.cc:492] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:45.136138 28798 ts_tablet_manager.cc:1403] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.000s
I20260618 13:59:45.136169 29011 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 143) took 3422 ms. Trace:
I20260618 13:59:45.136270 29011 rpcz_store.cc:276] 0618 13:59:41.713757 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:45.135709 (+3421952us) service_pool.cc:224] Handling call
0618 13:59:45.136138 (+ 429us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260618 13:59:45.136682 29011 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_a90110674b24430c976945f2749a2c40"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:45.137075 28798 raft_consensus.cc:359] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:45.137185 29011 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a90110674b24430c976945f2749a2c40 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:45.137218 28798 raft_consensus.cc:385] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.137262 28798 raft_consensus.cc:740] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.137455 28798 consensus_queue.cc:260] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.130589 29017 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 141) took 3426 ms. Trace:
I20260618 13:59:45.138191 29017 rpcz_store.cc:276] 0618 13:59:41.703685 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:45.081406 (+3377721us) service_pool.cc:224] Handling call
0618 13:59:45.130521 (+ 49115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260618 13:59:45.138594 29017 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_7438ba53b4e24b31a44bce5986fbf884"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:45.139153 29017 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7438ba53b4e24b31a44bce5986fbf884 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:45.140908 30900 raft_consensus.cc:493] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.141021 30900 raft_consensus.cc:515] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
W20260618 13:59:45.140137 29007 rpcz_store.cc:269] 0618 13:59:41.123794 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.141350 (+ 17556us) service_pool.cc:224] Handling call
0618 13:59:45.112188 (+3970838us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:43.074698 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:43.074793 (+ 95us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:43.074798 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:43.074800 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:43.074874 (+ 74us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:43.074883 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:43.074887 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:43.074947 (+ 60us) tablet.cc:701] Row locks acquired
0618 13:59:43.074949 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:43.074980 (+ 31us) write_op.cc:270] Start()
0618 13:59:43.075006 (+ 26us) write_op.cc:276] Timestamp: P: 1781791183074949 usec, L: 0
0618 13:59:43.075012 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:43.075200 (+ 188us) log.cc:844] Serialized 356 byte log entry
0618 13:59:43.163619 (+ 88419us) op_driver.cc:464] REPLICATION: finished
0618 13:59:43.164311 (+ 692us) write_op.cc:301] APPLY: starting
0618 13:59:43.164325 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:43.164360 (+ 35us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:43.164410 (+ 50us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:43.164455 (+ 45us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:43.164479 (+ 24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:43.164501 (+ 22us) write_op.cc:312] APPLY: finished
0618 13:59:43.164561 (+ 60us) log.cc:844] Serialized 23 byte log entry
0618 13:59:43.164604 (+ 43us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:43.164619 (+ 15us) write_op.cc:454] Released schema lock
0618 13:59:43.164626 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:43.164701 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:43.164705 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:43.844283 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:43.844436 (+ 153us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:43.844446 (+ 10us) write_op.cc:435] Acquired schema lock
0618 13:59:43.844451 (+ 5us) tablet.cc:662] Decoding operations
0618 13:59:43.844649 (+ 198us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:43.844671 (+ 22us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:43.844678 (+ 7us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:43.844777 (+ 99us) tablet.cc:701] Row locks acquired
0618 13:59:43.844782 (+ 5us) write_op.cc:260] PREPARE: finished
0618 13:59:43.844828 (+ 46us) write_op.cc:270] Start()
0618 13:59:43.844893 (+ 65us) write_op.cc:276] Timestamp: P: 1781791183844749 usec, L: 0
0618 13:59:43.844899 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:43.845135 (+ 236us) log.cc:844] Serialized 356 byte log entry
0618 13:59:44.001981 (+156846us) op_driver.cc:464] REPLICATION: finished
0618 13:59:44.016165 (+ 14184us) write_op.cc:301] APPLY: starting
0618 13:59:44.016185 (+ 20us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:44.016229 (+ 44us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:44.016273 (+ 44us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:44.016345 (+ 72us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:44.016378 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:44.016385 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:44.016471 (+ 86us) log.cc:844] Serialized 23 byte log entry
0618 13:59:44.016729 (+ 258us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.016750 (+ 21us) write_op.cc:454] Released schema lock
0618 13:59:44.016759 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:44.025064 (+ 8305us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:44.025074 (+ 10us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":80765,"reported_tablets":176,"child_traces":[["op",{"apply.queue_time_us":657,"apply.run_cpu_time_us":377,"apply.run_wall_time_us":376,"num_ops":1,"prepare.queue_time_us":597,"prepare.run_cpu_time_us":602,"prepare.run_wall_time_us":602,"raft.queue_time_us":3605,"raft.run_cpu_time_us":8,"raft.run_wall_time_us":9,"replication_time_us":88593}],["op",{"apply.queue_time_us":12447,"apply.run_cpu_time_us":512,"apply.run_wall_time_us":8882,"num_ops":1,"prepare.queue_time_us":95594,"prepare.run_cpu_time_us":1021,"prepare.run_wall_time_us":1820,"raft.queue_time_us":79,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":320,"replication_time_us":157027,"thread_start_us":120899,"threads_started":2}]]}
I20260618 13:59:45.141609 30900 leader_election.cc:290] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:45.142886 30950 ts_tablet_manager.cc:1929] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:45.144097 29007 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_e069e4a145044a9d908540306560f100"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:45.144757 29007 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e069e4a145044a9d908540306560f100 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:45.146919 28377 tablet_service.cc:1558] Processing DeleteTablet for tablet 72c04f55890648f0a8184f3783243f7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:38230
W20260618 13:59:45.143287 28484 leader_election.cc:343] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: d4667fe48f0f4eff9c6c8baabdab58d4
I20260618 13:59:45.147148 30950 log.cc:1199] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/98446f40214c4ac984fcfe98310005c5
I20260618 13:59:45.147164 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet ba2d6894bf5649fc8cfae7e033c432b9 with delete_type TABLET_DATA_DELETED (replaced by tablet a6ba4976bb6e4c2da3a563ccad4eb2b0) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:45.147513 29019 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 142) took 3433 ms. Trace:
I20260618 13:59:45.147888 29566 ts_tablet_manager.cc:1950] T 72c04f55890648f0a8184f3783243f7c P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:45.147624 29019 rpcz_store.cc:276] 0618 13:59:41.713662 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:45.098560 (+3384898us) service_pool.cc:224] Handling call
0618 13:59:45.133906 (+ 35346us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:45.134354 (+ 448us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:45.147278 (+ 12924us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:45.147381 (+ 103us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:45.147459 (+ 78us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:45.134898 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:45.135000 (+ 102us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:45.135007 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:45.135014 (+ 7us) tablet.cc:662] Decoding operations
0618 13:59:45.135100 (+ 86us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:45.135109 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:45.135114 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:45.135179 (+ 65us) tablet.cc:701] Row locks acquired
0618 13:59:45.135183 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:45.135219 (+ 36us) write_op.cc:270] Start()
0618 13:59:45.135246 (+ 27us) write_op.cc:276] Timestamp: P: 1781791185135184 usec, L: 0
0618 13:59:45.135250 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:45.135417 (+ 167us) log.cc:844] Serialized 433 byte log entry
0618 13:59:45.145457 (+ 10040us) op_driver.cc:464] REPLICATION: finished
0618 13:59:45.145574 (+ 117us) write_op.cc:301] APPLY: starting
0618 13:59:45.145587 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:45.145624 (+ 37us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:45.145654 (+ 30us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:45.145803 (+ 149us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:45.145833 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:45.145840 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:45.145902 (+ 62us) log.cc:844] Serialized 29 byte log entry
0618 13:59:45.145953 (+ 51us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.145967 (+ 14us) write_op.cc:454] Released schema lock
0618 13:59:45.145972 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:45.146052 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.146055 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":79,"apply.run_cpu_time_us":449,"apply.run_wall_time_us":448,"num_ops":2,"prepare.queue_time_us":67,"prepare.run_cpu_time_us":612,"prepare.run_wall_time_us":613,"raft.queue_time_us":364,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":10170}]]}
I20260618 13:59:45.148275 28384 tablet_service.cc:1558] Processing DeleteTablet for tablet c5d046e0421142cd84c5b12d6a434f53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:43 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:45.148422 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 2128dbec33db453ba51070767a15b21e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:45.148365 29019 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_fc6ead641f8e40de8744701bf5337457"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:45.148461 28377 tablet_service.cc:1558] Processing DeleteTablet for tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:38230
W20260618 13:59:45.148842 29019 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fc6ead641f8e40de8744701bf5337457 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:45.147388 28483 leader_election.cc:343] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: d4667fe48f0f4eff9c6c8baabdab58d4
I20260618 13:59:45.150104 28483 leader_election.cc:304] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:45.150519 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 319) took 3865 ms. Trace:
I20260618 13:59:45.150597 29566 rpcz_store.cc:276] 0618 13:59:41.285146 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.285764 (+ 618us) service_pool.cc:224] Handling call
0618 13:59:45.086323 (+3800559us) ts_tablet_manager.cc:1151] Deleting tablet 72c04f55890648f0a8184f3783243f7c
0618 13:59:45.086331 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.106271 (+ 19940us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:45.130904 (+ 24633us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:45.150500 (+ 19596us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3761262}
I20260618 13:59:45.151052 30900 raft_consensus.cc:2749] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:45.147693 28537 raft_consensus.cc:3060] T d4539f767c054e9e8ebc1f3f56108e7e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.144174 28798 ts_tablet_manager.cc:1434] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260618 13:59:45.151715 28798 tablet_bootstrap.cc:492] T 6c4a31a4f202473d97b17b5b5dfdb615 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:45.152118 30950 ts_tablet_manager.cc:1950] T 98446f40214c4ac984fcfe98310005c5 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:45.155893 30950 tablet_replica.cc:333] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:45.157531 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 72c04f55890648f0a8184f3783243f7c (table default.Table_773be314d7e741fcb4fb6eb7d954802e [id=0d83fc68255c458587390c58b12e00d7]) successfully deleted
I20260618 13:59:45.156960 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 323) took 3865 ms. Trace:
I20260618 13:59:45.153800 28798 tablet_bootstrap.cc:654] T 6c4a31a4f202473d97b17b5b5dfdb615 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.158640 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c40fe72e00a447582a35611de8ffa7c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:45.159322 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c40fe72e00a447582a35611de8ffa7c" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:45.158146 29566 rpcz_store.cc:276] 0618 13:59:41.291872 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.291919 (+ 47us) service_pool.cc:224] Handling call
0618 13:59:45.156876 (+3864957us) ts_tablet_manager.cc:1151] Deleting tablet d7a97176f72349cda221baaea61d18ad
0618 13:59:45.156884 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.156945 (+ 61us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3831797}
I20260618 13:59:45.159479 29017 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 145) took 3437 ms. Trace:
I20260618 13:59:45.161010 29017 rpcz_store.cc:276] 0618 13:59:41.722177 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:45.138368 (+3416191us) service_pool.cc:224] Handling call
0618 13:59:45.139277 (+ 909us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:45.139690 (+ 413us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:45.159215 (+ 19525us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:45.159322 (+ 107us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:45.159418 (+ 96us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:45.142764 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:45.142850 (+ 86us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:45.142855 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:45.142857 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:45.142929 (+ 72us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:45.142959 (+ 30us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:45.142964 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:45.143026 (+ 62us) tablet.cc:701] Row locks acquired
0618 13:59:45.143028 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:45.143068 (+ 40us) write_op.cc:270] Start()
0618 13:59:45.143092 (+ 24us) write_op.cc:276] Timestamp: P: 1781791185143031 usec, L: 0
0618 13:59:45.143096 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:45.143223 (+ 127us) log.cc:844] Serialized 433 byte log entry
0618 13:59:45.158119 (+ 14896us) op_driver.cc:464] REPLICATION: finished
0618 13:59:45.158810 (+ 691us) write_op.cc:301] APPLY: starting
0618 13:59:45.158820 (+ 10us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:45.158849 (+ 29us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:45.158880 (+ 31us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:45.158965 (+ 85us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:45.159000 (+ 35us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:45.159007 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:45.159073 (+ 66us) log.cc:844] Serialized 29 byte log entry
0618 13:59:45.159118 (+ 45us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.159135 (+ 17us) write_op.cc:454] Released schema lock
0618 13:59:45.159141 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":606,"apply.run_cpu_time_us":392,"apply.run_wall_time_us":393,"num_ops":2,"prepare.queue_time_us":2687,"prepare.run_cpu_time_us":540,"prepare.run_wall_time_us":540,"raft.queue_time_us":67,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":4,"replication_time_us":15003}]]}
I20260618 13:59:45.161453 30960 tablet_bootstrap.cc:492] T 8ac84641d0f1422193e42a42468789e1 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:45.161886 29017 master_service.cc:960] ReplaceTablet: received request to replace tablet 60b6216474b74a14af6abf977e312b77 from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:45.159960 29011 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 144) took 3444 ms. Trace:
I20260618 13:59:45.162206 29011 rpcz_store.cc:276] 0618 13:59:41.715409 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:45.136480 (+3421071us) service_pool.cc:224] Handling call
0618 13:59:45.137274 (+ 794us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:45.137681 (+ 407us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:45.159763 (+ 22082us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:45.159847 (+ 84us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:45.159924 (+ 77us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:45.139486 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:45.139568 (+ 82us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:45.139572 (+ 4us) write_op.cc:435] Acquired schema lock
0618 13:59:45.139575 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:45.139658 (+ 83us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:45.139668 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:45.139673 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:45.139758 (+ 85us) tablet.cc:701] Row locks acquired
0618 13:59:45.139760 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:45.139791 (+ 31us) write_op.cc:270] Start()
0618 13:59:45.139818 (+ 27us) write_op.cc:276] Timestamp: P: 1781791185139756 usec, L: 0
0618 13:59:45.139821 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:45.139954 (+ 133us) log.cc:844] Serialized 433 byte log entry
0618 13:59:45.158030 (+ 18076us) op_driver.cc:464] REPLICATION: finished
0618 13:59:45.158243 (+ 213us) write_op.cc:301] APPLY: starting
0618 13:59:45.158257 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:45.158299 (+ 42us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:45.158324 (+ 25us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:45.158476 (+ 152us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:45.158508 (+ 32us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:45.158515 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:45.158583 (+ 68us) log.cc:844] Serialized 29 byte log entry
0618 13:59:45.158645 (+ 62us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.158665 (+ 20us) write_op.cc:454] Released schema lock
0618 13:59:45.158671 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:45.158741 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.158745 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":163,"apply.run_cpu_time_us":482,"apply.run_wall_time_us":489,"mutex_wait_us":2667,"num_ops":2,"prepare.queue_time_us":1341,"prepare.run_cpu_time_us":558,"prepare.run_wall_time_us":558,"raft.queue_time_us":329,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":7,"replication_time_us":18180}]]}
I20260618 13:59:45.162909 30960 tablet_bootstrap.cc:654] T 8ac84641d0f1422193e42a42468789e1 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.163225 29011 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_e2020f3cb00e47e88950c4a8883e0ae4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:45.164397 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 8fea86ca38a741238a59053151087f69 because the tablet was not found. No further retry: Not found: Tablet not found: 8fea86ca38a741238a59053151087f69
I20260618 13:59:45.163856 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 325) took 3871 ms. Trace:
W20260618 13:59:45.164675 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet fea86899d2c047d997cebe80551e83c9 because the tablet was not found. No further retry: Not found: Tablet not found: fea86899d2c047d997cebe80551e83c9
I20260618 13:59:45.164616 29566 rpcz_store.cc:276] 0618 13:59:41.292362 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.292417 (+ 55us) service_pool.cc:224] Handling call
0618 13:59:45.163779 (+3871362us) ts_tablet_manager.cc:1151] Deleting tablet 751cd9bd961b4f56a588ed02a05f0212
0618 13:59:45.163788 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.163842 (+ 54us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3838609}
W20260618 13:59:45.164753 29011 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e2020f3cb00e47e88950c4a8883e0ae4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:45.164959 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 72c04f55890648f0a8184f3783243f7c because the tablet was not found. No further retry: Not found: Tablet not found: 72c04f55890648f0a8184f3783243f7c
I20260618 13:59:45.165001 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 329) took 3871 ms. Trace:
I20260618 13:59:45.165076 29566 rpcz_store.cc:276] 0618 13:59:41.293857 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.294329 (+ 472us) service_pool.cc:224] Handling call
0618 13:59:45.164926 (+3870597us) ts_tablet_manager.cc:1151] Deleting tablet d667ea5f029f46d89fcb3be03069b97b
0618 13:59:45.164936 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.164988 (+ 52us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3839566}
W20260618 13:59:45.165171 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet ba2d6894bf5649fc8cfae7e033c432b9 because the tablet was not found. No further retry: Not found: Tablet not found: ba2d6894bf5649fc8cfae7e033c432b9
W20260618 13:59:45.167479 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d7a97176f72349cda221baaea61d18ad because the tablet was not found. No further retry: Not found: Tablet not found: d7a97176f72349cda221baaea61d18ad
W20260618 13:59:45.168058 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 751cd9bd961b4f56a588ed02a05f0212 because the tablet was not found. No further retry: Not found: Tablet not found: 751cd9bd961b4f56a588ed02a05f0212
I20260618 13:59:45.161795 28543 raft_consensus.cc:3060] T e3a3395aa90e426eb48a3920a8c45ba8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:45.168757 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d667ea5f029f46d89fcb3be03069b97b because the tablet was not found. No further retry: Not found: Tablet not found: d667ea5f029f46d89fcb3be03069b97b
I20260618 13:59:45.173451 29007 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 146) took 3450 ms. Trace:
I20260618 13:59:45.173961 29007 rpcz_store.cc:276] 0618 13:59:41.722587 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:45.143934 (+3421347us) service_pool.cc:224] Handling call
0618 13:59:45.146122 (+ 2188us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:45.146641 (+ 519us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:45.173234 (+ 26593us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:45.173331 (+ 97us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:45.173410 (+ 79us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:45.156068 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:45.156235 (+ 167us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:45.156244 (+ 9us) write_op.cc:435] Acquired schema lock
0618 13:59:45.156248 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:45.156359 (+ 111us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:45.156372 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:45.156380 (+ 8us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:45.156475 (+ 95us) tablet.cc:701] Row locks acquired
0618 13:59:45.156480 (+ 5us) write_op.cc:260] PREPARE: finished
0618 13:59:45.156519 (+ 39us) write_op.cc:270] Start()
0618 13:59:45.156558 (+ 39us) write_op.cc:276] Timestamp: P: 1781791185156479 usec, L: 0
0618 13:59:45.156577 (+ 19us) op_driver.cc:348] REPLICATION: starting
0618 13:59:45.156771 (+ 194us) log.cc:844] Serialized 433 byte log entry
0618 13:59:45.170462 (+ 13691us) op_driver.cc:464] REPLICATION: finished
0618 13:59:45.170661 (+ 199us) write_op.cc:301] APPLY: starting
0618 13:59:45.170674 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:45.170707 (+ 33us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:45.170728 (+ 21us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:45.170823 (+ 95us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:45.170853 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:45.170859 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:45.170927 (+ 68us) log.cc:844] Serialized 29 byte log entry
0618 13:59:45.170983 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.170997 (+ 14us) write_op.cc:454] Released schema lock
0618 13:59:45.171003 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:45.171092 (+ 89us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.171096 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":149,"apply.run_cpu_time_us":401,"apply.run_wall_time_us":400,"num_ops":2,"prepare.queue_time_us":9011,"prepare.run_cpu_time_us":782,"prepare.run_wall_time_us":782,"replication_time_us":13833}]]}
I20260618 13:59:45.174374 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 332) took 3792 ms. Trace:
I20260618 13:59:45.175215 29566 rpcz_store.cc:276] 0618 13:59:41.381897 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.382110 (+ 213us) service_pool.cc:224] Handling call
0618 13:59:45.174296 (+3792186us) ts_tablet_manager.cc:1151] Deleting tablet 0463fa2e30554abcbfc8f37b702e81d4
0618 13:59:45.174305 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.174359 (+ 54us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3792002}
I20260618 13:59:45.175735 29007 master_service.cc:960] ReplaceTablet: received request to replace tablet c9ceb3af08fa4618b81cf6f89d7bd0df from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:45.176121 28798 tablet_bootstrap.cc:492] T 6c4a31a4f202473d97b17b5b5dfdb615 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:45.176281 29566 tablet_replica.cc:333] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:45.176287 28798 ts_tablet_manager.cc:1403] T 6c4a31a4f202473d97b17b5b5dfdb615 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.003s
I20260618 13:59:45.176409 30900 raft_consensus.cc:493] T a3c74d3ca66f42ad83ef54d9459b99fd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.176493 29566 raft_consensus.cc:2243] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:45.177000 28798 raft_consensus.cc:359] T 6c4a31a4f202473d97b17b5b5dfdb615 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.177181 28798 raft_consensus.cc:385] T 6c4a31a4f202473d97b17b5b5dfdb615 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.177251 28798 raft_consensus.cc:740] T 6c4a31a4f202473d97b17b5b5dfdb615 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.177475 28798 consensus_queue.cc:260] T 6c4a31a4f202473d97b17b5b5dfdb615 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.178041 28798 ts_tablet_manager.cc:1434] T 6c4a31a4f202473d97b17b5b5dfdb615 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:45.178292 28798 tablet_bootstrap.cc:492] T 10e9a417eecf4d33ac3bd9bd6325f447 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:45.178524 29566 raft_consensus.cc:2272] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.179059 29566 ts_tablet_manager.cc:1916] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.179067 30552 raft_consensus.cc:493] T 3f5d4296952e48a48756e6f27140da87 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.179162 30552 raft_consensus.cc:515] T 3f5d4296952e48a48756e6f27140da87 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.179718 30552 leader_election.cc:290] T 3f5d4296952e48a48756e6f27140da87 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.180143 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f5d4296952e48a48756e6f27140da87" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:45.180495 28798 tablet_bootstrap.cc:654] T 10e9a417eecf4d33ac3bd9bd6325f447 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.182001 30622 raft_consensus.cc:493] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.182155 30622 raft_consensus.cc:515] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.183040 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f5d4296952e48a48756e6f27140da87" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:45.183693 28484 leader_election.cc:304] T 3f5d4296952e48a48756e6f27140da87 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:45.183902 30552 raft_consensus.cc:2749] T 3f5d4296952e48a48756e6f27140da87 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:45.184340 30622 leader_election.cc:290] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:45.187160 28346 leader_election.cc:343] T 7c40fe72e00a447582a35611de8ffa7c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.191987 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779755a607bf46219900412b527e58b5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:45.192157 28682 raft_consensus.cc:2468] T 779755a607bf46219900412b527e58b5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:45.194586 28539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779755a607bf46219900412b527e58b5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:45.196918 28346 leader_election.cc:343] T 7c40fe72e00a447582a35611de8ffa7c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.197146 28346 leader_election.cc:304] T 7c40fe72e00a447582a35611de8ffa7c P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:45.197438 30152 raft_consensus.cc:2749] T 7c40fe72e00a447582a35611de8ffa7c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:45.198952 28346 consensus_peers.cc:597] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:45.176599 29011 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 149) took 3101 ms. Trace:
I20260618 13:59:45.199375 29011 rpcz_store.cc:276] 0618 13:59:42.075421 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:45.163059 (+3087638us) service_pool.cc:224] Handling call
0618 13:59:45.164828 (+ 1769us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:45.165318 (+ 490us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:45.176383 (+ 11065us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:45.176486 (+ 103us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:45.176563 (+ 77us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:45.165791 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:45.165909 (+ 118us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:45.165913 (+ 4us) write_op.cc:435] Acquired schema lock
0618 13:59:45.165915 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:45.165992 (+ 77us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:45.166000 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:45.166004 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:45.166067 (+ 63us) tablet.cc:701] Row locks acquired
0618 13:59:45.166069 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:45.166102 (+ 33us) write_op.cc:270] Start()
0618 13:59:45.166129 (+ 27us) write_op.cc:276] Timestamp: P: 1781791185166062 usec, L: 0
0618 13:59:45.166132 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:45.166274 (+ 142us) log.cc:844] Serialized 433 byte log entry
0618 13:59:45.174838 (+ 8564us) op_driver.cc:464] REPLICATION: finished
0618 13:59:45.175113 (+ 275us) write_op.cc:301] APPLY: starting
0618 13:59:45.175130 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:45.175172 (+ 42us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:45.175209 (+ 37us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:45.175352 (+ 143us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:45.175387 (+ 35us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:45.175398 (+ 11us) write_op.cc:312] APPLY: finished
0618 13:59:45.175460 (+ 62us) log.cc:844] Serialized 29 byte log entry
0618 13:59:45.175533 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.175551 (+ 18us) write_op.cc:454] Released schema lock
0618 13:59:45.175556 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:45.175618 (+ 62us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.175622 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":224,"apply.run_cpu_time_us":497,"apply.run_wall_time_us":497,"num_ops":2,"prepare.queue_time_us":44,"prepare.run_cpu_time_us":547,"prepare.run_wall_time_us":548,"replication_time_us":8670}]]}
I20260618 13:59:45.200434 29011 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_98d3bc8b4a7044d685c351a61e02f3e3" } new_table_name: "default.Table_3e7ca0884351432197632927d9c9db55" modify_external_catalogs: true
W20260618 13:59:45.178659 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 0463fa2e30554abcbfc8f37b702e81d4 because the tablet was not found. No further retry: Not found: Tablet not found: 0463fa2e30554abcbfc8f37b702e81d4
I20260618 13:59:45.201377 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 98446f40214c4ac984fcfe98310005c5 (table default.Table_83d021ed148243ffb453cb5e2484b7b4 [id=b3cae943ecdf410ca1cf711c5ddfe14a]) successfully deleted
I20260618 13:59:45.130757 29025 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 140) took 3464 ms. Trace:
I20260618 13:59:45.202029 29025 rpcz_store.cc:276] 0618 13:59:41.666138 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:45.066824 (+3400686us) service_pool.cc:224] Handling call
0618 13:59:45.074402 (+ 7578us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:45.074910 (+ 508us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:45.113164 (+ 38254us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:45.130630 (+ 17466us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:45.130718 (+ 88us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:45.097473 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:45.097594 (+ 121us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:45.097601 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:45.097603 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:45.097697 (+ 94us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:45.097710 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:45.097715 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:45.097795 (+ 80us) tablet.cc:701] Row locks acquired
0618 13:59:45.097798 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:45.097829 (+ 31us) write_op.cc:270] Start()
0618 13:59:45.097862 (+ 33us) write_op.cc:276] Timestamp: P: 1781791185097798 usec, L: 0
0618 13:59:45.097865 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:45.098031 (+ 166us) log.cc:844] Serialized 433 byte log entry
0618 13:59:45.102036 (+ 4005us) op_driver.cc:464] REPLICATION: finished
0618 13:59:45.102178 (+ 142us) write_op.cc:301] APPLY: starting
0618 13:59:45.102193 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:45.102240 (+ 47us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:45.102275 (+ 35us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:45.102392 (+ 117us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:45.102447 (+ 55us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:45.102457 (+ 10us) write_op.cc:312] APPLY: finished
0618 13:59:45.102534 (+ 77us) log.cc:844] Serialized 29 byte log entry
0618 13:59:45.102594 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.102614 (+ 20us) write_op.cc:454] Released schema lock
0618 13:59:45.102620 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:45.102687 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.102691 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":82,"apply.run_cpu_time_us":500,"apply.run_wall_time_us":499,"num_ops":2,"prepare.queue_time_us":50,"prepare.run_cpu_time_us":701,"prepare.run_wall_time_us":701,"raft.queue_time_us":206,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":226,"replication_time_us":4137}]]}
W20260618 13:59:45.203305 28346 leader_election.cc:343] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.204279 29025 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_5592094c6a024bd6ac90af5c18b0daab" } new_table_name: "default.Table_ee2255e7c482462580209cfe2d6fd9b5" modify_external_catalogs: true
I20260618 13:59:45.205271 30152 consensus_queue.cc:1048] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:45.205881 28346 leader_election.cc:304] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.206337 30152 raft_consensus.cc:2804] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:45.206406 30152 raft_consensus.cc:493] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:45.206485 30152 raft_consensus.cc:3060] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.158965 30950 raft_consensus.cc:2243] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:45.209331 30950 raft_consensus.cc:2272] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.209873 30950 ts_tablet_manager.cc:1916] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.185290 29024 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 91d014b46a0f448b929cbc9287ead0a7
W20260618 13:59:45.190611 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet c5d046e0421142cd84c5b12d6a434f53 because the tablet was not found. No further retry: Not found: Tablet not found: c5d046e0421142cd84c5b12d6a434f53
W20260618 13:59:45.212815 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 2128dbec33db453ba51070767a15b21e because the tablet was not found. No further retry: Not found: Tablet not found: 2128dbec33db453ba51070767a15b21e
W20260618 13:59:45.213038 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 because the tablet was not found. No further retry: Not found: Tablet not found: 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3
I20260618 13:59:45.213862 29024 catalog_manager.cc:6784] ReplaceTablet: tablet 91d014b46a0f448b929cbc9287ead0a7 deleted and replaced by tablet aa7faa3bb1b7474cb84e5f2a96af65e8
I20260618 13:59:45.213971 29024 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 138) took 3615 ms. Trace:
I20260618 13:59:45.214072 29024 rpcz_store.cc:276] 0618 13:59:41.598181 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:45.031689 (+3433508us) service_pool.cc:224] Handling call
0618 13:59:45.185020 (+153331us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.185027 (+ 7us) write_op.cc:454] Released schema lock
0618 13:59:45.213939 (+ 28912us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:45.131841 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:45.131972 (+ 131us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:45.131979 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:45.131982 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:45.132131 (+ 149us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:45.132146 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:45.132151 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:45.132235 (+ 84us) tablet.cc:701] Row locks acquired
0618 13:59:45.132238 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:45.132275 (+ 37us) write_op.cc:270] Start()
0618 13:59:45.132309 (+ 34us) write_op.cc:276] Timestamp: P: 1781791185132239 usec, L: 0
0618 13:59:45.132314 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:45.132495 (+ 181us) log.cc:844] Serialized 659 byte log entry
0618 13:59:45.137534 (+ 5039us) op_driver.cc:464] REPLICATION: finished
0618 13:59:45.137693 (+ 159us) write_op.cc:301] APPLY: starting
0618 13:59:45.137707 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:45.137752 (+ 45us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:45.137775 (+ 23us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:45.137875 (+ 100us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:45.137953 (+ 78us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:45.137959 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:45.138034 (+ 75us) log.cc:844] Serialized 29 byte log entry
0618 13:59:45.138127 (+ 93us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.138144 (+ 17us) write_op.cc:454] Released schema lock
0618 13:59:45.138151 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":112,"apply.run_cpu_time_us":518,"apply.run_wall_time_us":517,"num_ops":2,"prepare.queue_time_us":432,"prepare.run_cpu_time_us":770,"prepare.run_wall_time_us":769,"raft.queue_time_us":846,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":234,"replication_time_us":5194}]]}
I20260618 13:59:45.215029 28645 tablet_service.cc:1558] Processing DeleteTablet for tablet 91d014b46a0f448b929cbc9287ead0a7 with delete_type TABLET_DATA_DELETED (replaced by tablet aa7faa3bb1b7474cb84e5f2a96af65e8) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:45.215680 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet 91d014b46a0f448b929cbc9287ead0a7 with delete_type TABLET_DATA_DELETED (replaced by tablet aa7faa3bb1b7474cb84e5f2a96af65e8) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.218185 28797 tablet_bootstrap.cc:492] T 5ffd607dd9eb43e0a3769fcdd28ed05f P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:45.221572 28797 ts_tablet_manager.cc:1403] T 5ffd607dd9eb43e0a3769fcdd28ed05f P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.105s user 0.002s sys 0.002s
I20260618 13:59:45.224565 28797 raft_consensus.cc:359] T 5ffd607dd9eb43e0a3769fcdd28ed05f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.225322 28797 raft_consensus.cc:385] T 5ffd607dd9eb43e0a3769fcdd28ed05f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.225518 28797 raft_consensus.cc:740] T 5ffd607dd9eb43e0a3769fcdd28ed05f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.225888 28797 consensus_queue.cc:260] T 5ffd607dd9eb43e0a3769fcdd28ed05f P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.226400 30152 raft_consensus.cc:515] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.227181 30152 leader_election.cc:290] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:45.230350 28798 tablet_bootstrap.cc:492] T 10e9a417eecf4d33ac3bd9bd6325f447 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:45.230516 28798 ts_tablet_manager.cc:1403] T 10e9a417eecf4d33ac3bd9bd6325f447 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.052s user 0.005s sys 0.004s
I20260618 13:59:45.231225 28798 raft_consensus.cc:359] T 10e9a417eecf4d33ac3bd9bd6325f447 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.231431 28798 raft_consensus.cc:385] T 10e9a417eecf4d33ac3bd9bd6325f447 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.231490 28798 raft_consensus.cc:740] T 10e9a417eecf4d33ac3bd9bd6325f447 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.231720 28798 consensus_queue.cc:260] T 10e9a417eecf4d33ac3bd9bd6325f447 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.232270 28798 ts_tablet_manager.cc:1434] T 10e9a417eecf4d33ac3bd9bd6325f447 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:45.232577 28798 tablet_bootstrap.cc:492] T fea86899d2c047d997cebe80551e83c9 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:45.233477 30152 raft_consensus.cc:493] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.233597 30152 raft_consensus.cc:515] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.233790 28730 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:45.234143 28798 tablet_bootstrap.cc:654] T fea86899d2c047d997cebe80551e83c9 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.234146 30152 leader_election.cc:290] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:45.234956 29024 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_84a1abff52a348629514e93e578bfac3" } new_table_name: "default.Table_bebdea167bb0486d89b2043d0c8351e5" modify_external_catalogs: true
I20260618 13:59:45.240278 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779755a607bf46219900412b527e58b5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:45.240394 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779755a607bf46219900412b527e58b5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:45.240600 28680 raft_consensus.cc:3060] T 779755a607bf46219900412b527e58b5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.243749 28680 raft_consensus.cc:2468] T 779755a607bf46219900412b527e58b5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:45.246019 30152 raft_consensus.cc:493] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.246240 30960 tablet_bootstrap.cc:492] T 8ac84641d0f1422193e42a42468789e1 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:45.246217 30152 raft_consensus.cc:515] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.246445 30960 ts_tablet_manager.cc:1403] T 8ac84641d0f1422193e42a42468789e1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.085s user 0.005s sys 0.010s
I20260618 13:59:45.246832 30847 consensus_queue.cc:1048] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:45.247166 30960 raft_consensus.cc:359] T 8ac84641d0f1422193e42a42468789e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.247226 30152 leader_election.cc:290] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:45.248226 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 91d014b46a0f448b929cbc9287ead0a7 with delete_type TABLET_DATA_DELETED (replaced by tablet aa7faa3bb1b7474cb84e5f2a96af65e8) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:45.248428 30939 tablet_replica.cc:333] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:45.248586 30939 raft_consensus.cc:2243] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:45.248564 30552 raft_consensus.cc:493] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.248723 30939 raft_consensus.cc:2272] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.248683 30552 raft_consensus.cc:515] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.249214 30939 ts_tablet_manager.cc:1916] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.249215 30552 leader_election.cc:290] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:45.250129 28483 leader_election.cc:343] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: e3d8616846c649b6a969b971db0d03db
W20260618 13:59:45.255939 28983 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.60:42083, user_credentials={real_user=slave}} blocked reactor thread for 61886.4us
W20260618 13:59:45.258106 28983 consensus_peers.cc:597] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0 -> Peer 7637285570574a6dbcef1de6f84ecac0 (127.27.100.60:42083): Couldn't send request to peer 7637285570574a6dbcef1de6f84ecac0. Status: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260618 13:59:45.259057 28797 ts_tablet_manager.cc:1434] T 5ffd607dd9eb43e0a3769fcdd28ed05f P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.036s user 0.001s sys 0.001s
I20260618 13:59:45.259368 28797 tablet_bootstrap.cc:492] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:45.266073 28797 tablet_bootstrap.cc:654] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.268328 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3d8616846c649b6a969b971db0d03db" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:45.268532 28680 raft_consensus.cc:2468] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 0.
I20260618 13:59:45.269214 28484 leader_election.cc:304] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:45.269445 30552 raft_consensus.cc:2804] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:45.269464 29007 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet c9ceb3af08fa4618b81cf6f89d7bd0df
I20260618 13:59:45.269507 30552 raft_consensus.cc:493] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:45.269634 30552 raft_consensus.cc:3060] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.270015 29007 catalog_manager.cc:6784] ReplaceTablet: tablet c9ceb3af08fa4618b81cf6f89d7bd0df deleted and replaced by tablet a1896e3b4a904fef8a741f63aa952251
I20260618 13:59:45.270154 29007 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 150) took 3185 ms. Trace:
I20260618 13:59:45.270238 29007 rpcz_store.cc:276] 0618 13:59:42.084975 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:45.175646 (+3090671us) service_pool.cc:224] Handling call
0618 13:59:45.270097 (+ 94451us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:45.240672 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:45.240800 (+ 128us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:45.240805 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:45.240807 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:45.240924 (+ 117us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:45.240936 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:45.240942 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:45.241053 (+ 111us) tablet.cc:701] Row locks acquired
0618 13:59:45.241057 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:45.241113 (+ 56us) write_op.cc:270] Start()
0618 13:59:45.241147 (+ 34us) write_op.cc:276] Timestamp: P: 1781791185241063 usec, L: 0
0618 13:59:45.241151 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:45.241326 (+ 175us) log.cc:844] Serialized 625 byte log entry
0618 13:59:45.268488 (+ 27162us) op_driver.cc:464] REPLICATION: finished
0618 13:59:45.268772 (+ 284us) write_op.cc:301] APPLY: starting
0618 13:59:45.268787 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:45.268828 (+ 41us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:45.268853 (+ 25us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:45.268956 (+ 103us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:45.269010 (+ 54us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:45.269017 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:45.269098 (+ 81us) log.cc:844] Serialized 29 byte log entry
0618 13:59:45.269158 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.269175 (+ 17us) write_op.cc:454] Released schema lock
0618 13:59:45.269181 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:45.269256 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.269260 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":242,"apply.run_cpu_time_us":474,"apply.run_wall_time_us":474,"num_ops":2,"prepare.queue_time_us":15690,"prepare.run_cpu_time_us":715,"prepare.run_wall_time_us":981,"replication_time_us":27307}]]}
I20260618 13:59:45.271654 28645 tablet_service.cc:1558] Processing DeleteTablet for tablet c9ceb3af08fa4618b81cf6f89d7bd0df with delete_type TABLET_DATA_DELETED (replaced by tablet a1896e3b4a904fef8a741f63aa952251) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:45.272357 30552 raft_consensus.cc:515] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.273036 29019 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 147) took 3548 ms. Trace:
I20260618 13:59:45.273160 29019 rpcz_store.cc:276] 0618 13:59:41.724489 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:45.148184 (+3423695us) service_pool.cc:224] Handling call
0618 13:59:45.148935 (+ 751us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:45.149470 (+ 535us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:45.272704 (+123234us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:45.272839 (+ 135us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:45.272960 (+ 121us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:45.241684 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:45.241776 (+ 92us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:45.241780 (+ 4us) write_op.cc:435] Acquired schema lock
0618 13:59:45.241783 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:45.241848 (+ 65us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:45.241856 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:45.241861 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:45.241943 (+ 82us) tablet.cc:701] Row locks acquired
0618 13:59:45.241945 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:45.241972 (+ 27us) write_op.cc:270] Start()
0618 13:59:45.241994 (+ 22us) write_op.cc:276] Timestamp: P: 1781791185241927 usec, L: 0
0618 13:59:45.241997 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:45.242160 (+ 163us) log.cc:844] Serialized 433 byte log entry
0618 13:59:45.268562 (+ 26402us) op_driver.cc:464] REPLICATION: finished
0618 13:59:45.269325 (+ 763us) write_op.cc:301] APPLY: starting
0618 13:59:45.269336 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:45.269369 (+ 33us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:45.269389 (+ 20us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:45.269491 (+ 102us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:45.269517 (+ 26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:45.269523 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:45.269581 (+ 58us) log.cc:844] Serialized 29 byte log entry
0618 13:59:45.269621 (+ 40us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.269633 (+ 12us) write_op.cc:454] Released schema lock
0618 13:59:45.269637 (+ 4us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:45.269698 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.269703 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":730,"apply.run_cpu_time_us":365,"apply.run_wall_time_us":364,"mutex_wait_us":80562,"num_ops":2,"prepare.queue_time_us":91779,"prepare.run_cpu_time_us":557,"prepare.run_wall_time_us":824,"replication_time_us":26550}]]}
W20260618 13:59:45.273893 28483 leader_election.cc:343] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: e3d8616846c649b6a969b971db0d03db
I20260618 13:59:45.274031 30552 leader_election.cc:290] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Requested vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.274330 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet c9ceb3af08fa4618b81cf6f89d7bd0df with delete_type TABLET_DATA_DELETED (replaced by tablet a1896e3b4a904fef8a741f63aa952251) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.276142 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:45.277786 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:45.273887 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3d8616846c649b6a969b971db0d03db" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:45.278695 28680 raft_consensus.cc:3060] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.282198 28680 raft_consensus.cc:2468] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97859748af2c41879ba2a668dda999d2 in term 1.
I20260618 13:59:45.282701 28484 leader_election.cc:304] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b
I20260618 13:59:45.282927 30552 raft_consensus.cc:2804] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:45.284559 30900 raft_consensus.cc:515] T a3c74d3ca66f42ad83ef54d9459b99fd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.285732 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c74d3ca66f42ad83ef54d9459b99fd" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:45.286145 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3c74d3ca66f42ad83ef54d9459b99fd" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:45.286168 29019 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a" } modify_external_catalogs: true
I20260618 13:59:45.286446 29019 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a" } modify_external_catalogs: true
I20260618 13:59:45.286964 29007 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_bb918931281942b486860c5e4f5755f1" } modify_external_catalogs: true
I20260618 13:59:45.287120 29007 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_bb918931281942b486860c5e4f5755f1" } modify_external_catalogs: true
I20260618 13:59:45.288076 28484 leader_election.cc:304] T a3c74d3ca66f42ad83ef54d9459b99fd P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:45.291038 30939 ts_tablet_manager.cc:1929] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:45.291188 30939 log.cc:1199] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/91d014b46a0f448b929cbc9287ead0a7
I20260618 13:59:45.291632 30939 ts_tablet_manager.cc:1950] T 91d014b46a0f448b929cbc9287ead0a7 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:45.292263 28797 tablet_bootstrap.cc:492] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:45.292452 28797 ts_tablet_manager.cc:1403] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.003s
I20260618 13:59:45.294703 30950 ts_tablet_manager.cc:1929] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:45.294807 30950 log.cc:1199] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/c01c8463055e4526b3c75190066f30b6
I20260618 13:59:45.295127 30950 ts_tablet_manager.cc:1950] T c01c8463055e4526b3c75190066f30b6 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:45.295953 30950 tablet_replica.cc:333] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:45.297039 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:45.297343 28798 tablet_bootstrap.cc:492] T fea86899d2c047d997cebe80551e83c9 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:45.297483 28798 ts_tablet_manager.cc:1403] T fea86899d2c047d997cebe80551e83c9 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.065s user 0.010s sys 0.012s
I20260618 13:59:45.297457 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet c01c8463055e4526b3c75190066f30b6 (table default.Table_7e5d58ad3e8043919b66e8890819535c [id=96aff34b68c1494f9228a11eb38de4c7]) successfully deleted
I20260618 13:59:45.297649 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "732c8801c319400283806aac7f15917e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:45.297658 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "732c8801c319400283806aac7f15917e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:45.298034 30900 leader_election.cc:290] T a3c74d3ca66f42ad83ef54d9459b99fd P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.298239 30900 raft_consensus.cc:2749] T a3c74d3ca66f42ad83ef54d9459b99fd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:45.299503 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffd607dd9eb43e0a3769fcdd28ed05f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:45.299753 28680 raft_consensus.cc:2468] T 5ffd607dd9eb43e0a3769fcdd28ed05f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:45.300119 28797 raft_consensus.cc:359] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.300269 28797 raft_consensus.cc:385] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.300316 28797 raft_consensus.cc:740] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.300511 28797 consensus_queue.cc:260] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.301486 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 91d014b46a0f448b929cbc9287ead0a7 (table default.Table_62bf5ce52fb14bd089678663dd009a91 [id=713af0f85a434ed0bfb791c48f75dc9d]) successfully deleted
W20260618 13:59:45.301568 28346 consensus_peers.cc:597] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:45.301942 28797 ts_tablet_manager.cc:1434] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260618 13:59:45.302043 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffd607dd9eb43e0a3769fcdd28ed05f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:45.301726 28346 leader_election.cc:343] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:45.302613 28346 leader_election.cc:343] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:45.302793 28346 leader_election.cc:343] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.303227 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:45.304137 28346 leader_election.cc:304] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.304627 30622 raft_consensus.cc:2804] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:45.304747 30622 raft_consensus.cc:697] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:45.304978 30622 consensus_queue.cc:237] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.305218 29018 catalog_manager.cc:5696] T 8bef52c0fc354d6cb5388700641af9f8 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:45.247326 30960 raft_consensus.cc:385] T 8ac84641d0f1422193e42a42468789e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.306314 30960 raft_consensus.cc:740] T 8ac84641d0f1422193e42a42468789e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.306599 30960 consensus_queue.cc:260] T 8ac84641d0f1422193e42a42468789e1 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.307227 30960 ts_tablet_manager.cc:1434] T 8ac84641d0f1422193e42a42468789e1 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.061s user 0.000s sys 0.001s
I20260618 13:59:45.307363 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet e196dd3c7fe14524a50152643ddbc2ad (table default.Table_2dfebd62a146446f999f1a7a1bd1af79 [id=02ffed3d852845d6845fb5176f036fce]). Expected version 1 got 0
I20260618 13:59:45.308668 29566 ts_tablet_manager.cc:1929] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:45.308787 29566 log.cc:1199] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/b0d3d363766546c4a97b0763bad11f71
I20260618 13:59:45.308991 28797 tablet_bootstrap.cc:492] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:45.309190 29566 ts_tablet_manager.cc:1950] T b0d3d363766546c4a97b0763bad11f71 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:45.309600 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:45.309770 28798 raft_consensus.cc:359] T fea86899d2c047d997cebe80551e83c9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.309989 28798 raft_consensus.cc:385] T fea86899d2c047d997cebe80551e83c9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.310047 28798 raft_consensus.cc:740] T fea86899d2c047d997cebe80551e83c9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.310233 28798 consensus_queue.cc:260] T fea86899d2c047d997cebe80551e83c9 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.310657 28797 tablet_bootstrap.cc:654] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.311520 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:45.313549 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
W20260618 13:59:45.313964 28618 consensus_peers.cc:597] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:45.316154 28539 raft_consensus.cc:3060] T a3c74d3ca66f42ad83ef54d9459b99fd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.318819 30552 raft_consensus.cc:697] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Becoming Leader. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Running, Role: LEADER
I20260618 13:59:45.319264 30552 consensus_queue.cc:237] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.320665 28798 ts_tablet_manager.cc:1434] T fea86899d2c047d997cebe80551e83c9 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.023s user 0.002s sys 0.000s
I20260618 13:59:45.323076 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 341) took 3837 ms. Trace:
I20260618 13:59:45.323185 29566 rpcz_store.cc:276] 0618 13:59:41.485244 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.485307 (+ 63us) service_pool.cc:224] Handling call
0618 13:59:45.176231 (+3690924us) ts_tablet_manager.cc:1151] Deleting tablet b0d3d363766546c4a97b0763bad11f71
0618 13:59:45.176242 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.200671 (+ 24429us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:45.308606 (+107935us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:45.323039 (+ 14433us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3690756}
I20260618 13:59:45.324402 28797 tablet_bootstrap.cc:492] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:45.324553 28797 ts_tablet_manager.cc:1403] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.000s
I20260618 13:59:45.325269 28797 raft_consensus.cc:359] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.325445 28797 raft_consensus.cc:385] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.325503 28797 raft_consensus.cc:740] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.325730 28797 consensus_queue.cc:260] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.326212 28797 ts_tablet_manager.cc:1434] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:45.326517 28797 tablet_bootstrap.cc:492] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:45.323680 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet b0d3d363766546c4a97b0763bad11f71 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]) successfully deleted
I20260618 13:59:45.327410 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:45.328230 29018 catalog_manager.cc:5696] T c25fa765ec794c34b0aa43761961a8c9 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:45.328447 28798 tablet_bootstrap.cc:492] T 5115a31a38db48c1a6474917aff97c8e P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:45.329931 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:45.330224 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:45.330577 28798 tablet_bootstrap.cc:654] T 5115a31a38db48c1a6474917aff97c8e P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.330725 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_01e1e5741bb54f78996857f6b79fc5d7 [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 2 got 0
I20260618 13:59:45.331276 29018 catalog_manager.cc:5696] T a8da96fc621c47d4a444b5a39259f85d P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:45.332866 30950 raft_consensus.cc:2243] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:45.333266 30950 pending_rounds.cc:70] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2: Trying to abort 1 pending ops.
I20260618 13:59:45.333359 30950 pending_rounds.cc:77] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298216695071207424 op_type: NO_OP noop_request { }
I20260618 13:59:45.333504 30950 raft_consensus.cc:2889] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:45.333650 30950 raft_consensus.cc:2272] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.333842 28797 tablet_bootstrap.cc:654] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:45.334151 28346 leader_election.cc:343] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.334278 28346 leader_election.cc:304] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:45.334656 29566 tablet_replica.cc:333] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:45.334846 29566 raft_consensus.cc:2243] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:45.334951 30950 ts_tablet_manager.cc:1916] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.334985 29566 raft_consensus.cc:2272] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.335439 29566 ts_tablet_manager.cc:1916] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.338639 28346 leader_election.cc:304] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.339144 30622 raft_consensus.cc:2749] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:45.339214 30864 raft_consensus.cc:2804] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:45.339293 30864 raft_consensus.cc:493] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:45.339354 30864 raft_consensus.cc:3060] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.339821 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet c9ceb3af08fa4618b81cf6f89d7bd0df with delete_type TABLET_DATA_DELETED (replaced by tablet a1896e3b4a904fef8a741f63aa952251) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:45.340029 30939 tablet_replica.cc:333] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:45.340219 30939 raft_consensus.cc:2243] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:45.340332 30939 raft_consensus.cc:2272] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.340755 30939 ts_tablet_manager.cc:1916] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.341744 30864 raft_consensus.cc:515] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.342300 30864 leader_election.cc:290] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:45.345198 30950 ts_tablet_manager.cc:1929] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:45.345954 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 10e9a417eecf4d33ac3bd9bd6325f447 (table default.Table_21426dbf385d47d2af5fc25e287ed0ed [id=0de5461053eb4517b15d6b91875fc04d]). Expected version 1 got 0
I20260618 13:59:45.346614 30939 ts_tablet_manager.cc:1929] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260618 13:59:45.349943 28618 consensus_peers.cc:597] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:45.347545 28543 raft_consensus.cc:3060] T 10e9a417eecf4d33ac3bd9bd6325f447 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.353250 29566 ts_tablet_manager.cc:1929] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:45.355609 30854 raft_consensus.cc:493] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.355723 30854 raft_consensus.cc:515] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.356326 30854 leader_election.cc:290] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.356585 28798 tablet_bootstrap.cc:492] T 5115a31a38db48c1a6474917aff97c8e P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:45.356710 28798 ts_tablet_manager.cc:1403] T 5115a31a38db48c1a6474917aff97c8e P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.034s user 0.004s sys 0.000s
I20260618 13:59:45.357323 28539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffd607dd9eb43e0a3769fcdd28ed05f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:45.357784 28798 raft_consensus.cc:359] T 5115a31a38db48c1a6474917aff97c8e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.357807 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08132883fb4f4632af2961bead244b18" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:45.358033 28798 raft_consensus.cc:385] T 5115a31a38db48c1a6474917aff97c8e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.358099 28798 raft_consensus.cc:740] T 5115a31a38db48c1a6474917aff97c8e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.358314 28798 consensus_queue.cc:260] T 5115a31a38db48c1a6474917aff97c8e P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:45.358661 28618 leader_election.cc:343] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.358655 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ffd607dd9eb43e0a3769fcdd28ed05f" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:45.359016 28680 raft_consensus.cc:3060] T 5ffd607dd9eb43e0a3769fcdd28ed05f P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.359706 28798 ts_tablet_manager.cc:1434] T 5115a31a38db48c1a6474917aff97c8e P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260618 13:59:45.360500 30950 log.cc:1199] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/e3d8616846c649b6a969b971db0d03db
I20260618 13:59:45.362000 28680 raft_consensus.cc:2468] T 5ffd607dd9eb43e0a3769fcdd28ed05f P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:45.362532 30950 ts_tablet_manager.cc:1950] T e3d8616846c649b6a969b971db0d03db P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:45.351583 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 00b602a473a94c05a617fb9478300c74 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:45.363438 29018 catalog_manager.cc:5696] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:45.366055 28797 tablet_bootstrap.cc:492] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:45.366197 28797 ts_tablet_manager.cc:1403] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.040s user 0.001s sys 0.001s
I20260618 13:59:45.366434 30950 tablet_replica.cc:333] T fea86899d2c047d997cebe80551e83c9 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:45.366782 30950 raft_consensus.cc:2243] T fea86899d2c047d997cebe80551e83c9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:45.364606 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet e3d8616846c649b6a969b971db0d03db (table default.Table_473a0c23203b404ea8e5ee0aed201e7b [id=3656a2f8d6cc4892a071908ae55efd3f]) successfully deleted
I20260618 13:59:45.366963 28797 raft_consensus.cc:359] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.367152 28797 raft_consensus.cc:385] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.367269 28797 raft_consensus.cc:740] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.367470 28797 consensus_queue.cc:260] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.368381 28797 ts_tablet_manager.cc:1434] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:45.368713 30950 raft_consensus.cc:2272] T fea86899d2c047d997cebe80551e83c9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.369057 28797 tablet_bootstrap.cc:492] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:45.369481 29018 catalog_manager.cc:5696] T d8a71e334ed1400d97c965b49b677a81 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:45.369577 28539 raft_consensus.cc:3060] T 3f5d4296952e48a48756e6f27140da87 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.370790 28797 tablet_bootstrap.cc:654] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.373335 29566 log.cc:1199] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/8fea86ca38a741238a59053151087f69
I20260618 13:59:45.356834 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08132883fb4f4632af2961bead244b18" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:45.373687 29566 ts_tablet_manager.cc:1950] T 8fea86ca38a741238a59053151087f69 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:45.373667 28407 raft_consensus.cc:2468] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:45.370663 28346 leader_election.cc:343] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.374145 28617 leader_election.cc:304] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.374396 30856 raft_consensus.cc:2804] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:45.374502 30856 raft_consensus.cc:493] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:45.374585 30856 raft_consensus.cc:3060] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.377099 30856 raft_consensus.cc:515] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.377928 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 356) took 3525 ms. Trace:
I20260618 13:59:45.378108 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08132883fb4f4632af2961bead244b18" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:45.378016 29566 rpcz_store.cc:276] 0618 13:59:41.852193 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:41.852258 (+ 65us) service_pool.cc:224] Handling call
0618 13:59:45.334607 (+3482349us) ts_tablet_manager.cc:1151] Deleting tablet 8fea86ca38a741238a59053151087f69
0618 13:59:45.334617 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.350680 (+ 16063us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:45.353203 (+ 2523us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:45.377911 (+ 24708us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3482147}
I20260618 13:59:45.378552 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 8fea86ca38a741238a59053151087f69 (table default.Table_7ae70108b1714e5ca374cc653bed7b33 [id=0cf9d7859b7d4d95bf769070c43003d3]) successfully deleted
W20260618 13:59:45.378727 28618 leader_election.cc:343] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.379331 30856 leader_election.cc:290] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.379992 29018 catalog_manager.cc:5696] T 2e493f647a344cd581515a77230d765f P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:45.380302 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08132883fb4f4632af2961bead244b18" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:45.380472 28407 raft_consensus.cc:3060] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.360514 28798 tablet_bootstrap.cc:492] T a662bb37fb7d4df49e61e524808b67a8 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:45.383762 28407 raft_consensus.cc:2468] T 08132883fb4f4632af2961bead244b18 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:45.383948 28798 tablet_bootstrap.cc:654] T a662bb37fb7d4df49e61e524808b67a8 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.384109 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 371) took 2320 ms. Trace:
I20260618 13:59:45.384238 28617 leader_election.cc:304] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.384733 30854 raft_consensus.cc:2804] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:45.384855 30854 raft_consensus.cc:697] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:45.385169 30950 ts_tablet_manager.cc:1916] T fea86899d2c047d997cebe80551e83c9 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.385102 30854 consensus_queue.cc:237] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.374235 30622 consensus_queue.cc:1048] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260618 13:59:45.387593 28346 leader_election.cc:304] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.388252 30939 log.cc:1199] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/c9ceb3af08fa4618b81cf6f89d7bd0df
I20260618 13:59:45.388988 30890 consensus_queue.cc:1048] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:45.389035 30986 tablet.cc:1724] T 10e9a417eecf4d33ac3bd9bd6325f447 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:45.389451 30572 raft_consensus.cc:493] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.389559 30572 raft_consensus.cc:515] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.390061 30572 leader_election.cc:290] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.390758 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32d411b71d146059d40260e54c283ec" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:45.391242 28618 leader_election.cc:343] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:45.393129 28618 consensus_peers.cc:597] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:45.393986 28617 consensus_peers.cc:597] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 91d014b46a0f448b929cbc9287ead0a7. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:45.388464 30262 raft_consensus.cc:2804] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:45.394311 30262 raft_consensus.cc:697] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:45.394641 30262 consensus_queue.cc:237] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.391039 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32d411b71d146059d40260e54c283ec" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:45.395874 28407 raft_consensus.cc:2468] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:45.396298 28617 leader_election.cc:304] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.397198 30572 raft_consensus.cc:2804] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:45.397277 30572 raft_consensus.cc:493] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:45.397336 30572 raft_consensus.cc:3060] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.401144 30572 raft_consensus.cc:515] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.404698 30572 leader_election.cc:290] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.405270 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32d411b71d146059d40260e54c283ec" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:45.405704 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32d411b71d146059d40260e54c283ec" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:45.405949 28407 raft_consensus.cc:3060] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.406984 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:45.407303 28798 tablet_bootstrap.cc:492] T a662bb37fb7d4df49e61e524808b67a8 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:45.407464 28798 ts_tablet_manager.cc:1403] T a662bb37fb7d4df49e61e524808b67a8 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.047s user 0.007s sys 0.000s
I20260618 13:59:45.407488 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 8af2bec820b842b4bf6b57482007bd6a (table default.Table_84be6926f4b64876b343577b85db3243 [id=af1859c644b44c1d831314d27a494f94]). Expected version 1 got 0
W20260618 13:59:45.406056 28618 leader_election.cc:343] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.408126 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:45.408406 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:45.409077 29018 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:45.411127 28407 raft_consensus.cc:2468] T f32d411b71d146059d40260e54c283ec P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:45.411211 28797 tablet_bootstrap.cc:492] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:45.411345 28797 ts_tablet_manager.cc:1403] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.043s user 0.004s sys 0.002s
I20260618 13:59:45.411986 28797 raft_consensus.cc:359] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.412164 28797 raft_consensus.cc:385] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.412231 28797 raft_consensus.cc:740] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.412500 28797 consensus_queue.cc:260] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.413656 28797 ts_tablet_manager.cc:1434] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:45.413684 28617 leader_election.cc:304] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.413924 28797 tablet_bootstrap.cc:492] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:45.414746 30854 raft_consensus.cc:2804] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:45.414850 30854 raft_consensus.cc:697] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:45.415045 30854 consensus_queue.cc:237] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.416206 28798 raft_consensus.cc:359] T a662bb37fb7d4df49e61e524808b67a8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.416412 28798 raft_consensus.cc:385] T a662bb37fb7d4df49e61e524808b67a8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.416472 28798 raft_consensus.cc:740] T a662bb37fb7d4df49e61e524808b67a8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.417387 28798 consensus_queue.cc:260] T a662bb37fb7d4df49e61e524808b67a8 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.418766 28797 tablet_bootstrap.cc:654] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.418843 28798 ts_tablet_manager.cc:1434] T a662bb37fb7d4df49e61e524808b67a8 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260618 13:59:45.419091 28798 tablet_bootstrap.cc:492] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:45.420599 28798 tablet_bootstrap.cc:654] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.428972 30939 ts_tablet_manager.cc:1950] T c9ceb3af08fa4618b81cf6f89d7bd0df P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:45.431540 30552 raft_consensus.cc:493] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.431679 30552 raft_consensus.cc:515] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:45.432231 30552 leader_election.cc:290] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:45.432920 28483 leader_election.cc:343] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 962bf304a285412fb2a529db5b994e9e
W20260618 13:59:45.433542 28484 leader_election.cc:343] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 962bf304a285412fb2a529db5b994e9e
I20260618 13:59:45.433651 28484 leader_election.cc:304] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:45.433866 30552 raft_consensus.cc:2749] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:45.437769 28798 tablet_bootstrap.cc:492] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:45.437952 28798 ts_tablet_manager.cc:1403] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.002s
I20260618 13:59:45.439092 28798 raft_consensus.cc:359] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:45.440933 28618 consensus_peers.cc:597] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:45.443593 30950 ts_tablet_manager.cc:1929] T fea86899d2c047d997cebe80551e83c9 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:45.447309 30950 log.cc:1199] T fea86899d2c047d997cebe80551e83c9 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/fea86899d2c047d997cebe80551e83c9
I20260618 13:59:45.447671 30950 ts_tablet_manager.cc:1950] T fea86899d2c047d997cebe80551e83c9 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:45.448232 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet fea86899d2c047d997cebe80551e83c9 (table default.Table_539ac216b6f4478eb994218d7d7980a4 [id=baa3214376034e9cae926def4c46b91e]) successfully deleted
I20260618 13:59:45.448608 30950 tablet_replica.cc:333] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:45.448836 30950 raft_consensus.cc:2243] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:45.449107 30950 pending_rounds.cc:70] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2: Trying to abort 1 pending ops.
I20260618 13:59:45.449206 30950 pending_rounds.cc:77] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298216693031256064 op_type: NO_OP noop_request { }
I20260618 13:59:45.449347 30950 raft_consensus.cc:2889] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:45.449477 30950 raft_consensus.cc:2272] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.450670 30950 ts_tablet_manager.cc:1916] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.439813 28798 raft_consensus.cc:385] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.454653 28798 raft_consensus.cc:740] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.455090 28798 consensus_queue.cc:260] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.384196 29566 rpcz_store.cc:276] 0618 13:59:43.063214 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:43.063715 (+ 501us) service_pool.cc:224] Handling call
0618 13:59:45.383959 (+2320244us) ts_tablet_manager.cc:1151] Deleting tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3
0618 13:59:45.383972 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.384089 (+ 117us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2320029}
I20260618 13:59:45.455863 29566 tablet_replica.cc:333] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:45.456099 29566 raft_consensus.cc:2243] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:45.456247 29566 raft_consensus.cc:2272] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.456774 29566 ts_tablet_manager.cc:1916] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.457099 28798 ts_tablet_manager.cc:1434] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.019s user 0.001s sys 0.000s
W20260618 13:59:45.458947 28989 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 already in progress: creating tablet
I20260618 13:59:45.460827 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet c9ceb3af08fa4618b81cf6f89d7bd0df (table default.Table_21242eba3cb64c7c91082e296439cfe2 [id=10a4f5ef579746b68d8dd37d83f14c3e]) successfully deleted
I20260618 13:59:45.467945 30950 ts_tablet_manager.cc:1929] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:45.474673 30950 log.cc:1199] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/8fea86ca38a741238a59053151087f69
I20260618 13:59:45.476475 30950 ts_tablet_manager.cc:1950] T 8fea86ca38a741238a59053151087f69 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:45.477469 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 8fea86ca38a741238a59053151087f69 (table default.Table_7ae70108b1714e5ca374cc653bed7b33 [id=0cf9d7859b7d4d95bf769070c43003d3]) successfully deleted
W20260618 13:59:45.478513 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 97a9f949999446f09359629fe483bf67 because the tablet was not found. No further retry: Not found: Tablet not found: 97a9f949999446f09359629fe483bf67
W20260618 13:59:45.479358 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet e5f7eaa7d218440ca2c35340c5c87814 because the tablet was not found. No further retry: Not found: Tablet not found: e5f7eaa7d218440ca2c35340c5c87814
W20260618 13:59:45.480199 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet b0d3d363766546c4a97b0763bad11f71 because the tablet was not found. No further retry: Not found: Tablet not found: b0d3d363766546c4a97b0763bad11f71
I20260618 13:59:45.481921 28539 raft_consensus.cc:3060] T 9d8f124ba79b46e098303b7b2705844d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:45.482643 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 already in progress: creating tablet
W20260618 13:59:45.483456 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 2128dbec33db453ba51070767a15b21e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2128dbec33db453ba51070767a15b21e already in progress: creating tablet
I20260618 13:59:45.490294 28797 tablet_bootstrap.cc:492] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:45.490476 28797 ts_tablet_manager.cc:1403] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.077s user 0.002s sys 0.006s
I20260618 13:59:45.491430 28797 raft_consensus.cc:359] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.491655 28797 raft_consensus.cc:385] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.491716 28797 raft_consensus.cc:740] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.491927 28797 consensus_queue.cc:260] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.492506 28797 ts_tablet_manager.cc:1434] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:45.492877 28797 tablet_bootstrap.cc:492] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:45.494956 29017 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 60b6216474b74a14af6abf977e312b77
I20260618 13:59:45.495477 28797 tablet_bootstrap.cc:654] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.495509 29017 catalog_manager.cc:6784] ReplaceTablet: tablet 60b6216474b74a14af6abf977e312b77 deleted and replaced by tablet f683917aac5945ec939664ab82c24530
I20260618 13:59:45.495611 29017 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33144 (request call id 148) took 3717 ms. Trace:
I20260618 13:59:45.495807 30950 tablet_replica.cc:333] T 6c4a31a4f202473d97b17b5b5dfdb615 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:45.496014 30950 raft_consensus.cc:2243] T 6c4a31a4f202473d97b17b5b5dfdb615 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:45.496155 30950 raft_consensus.cc:2272] T 6c4a31a4f202473d97b17b5b5dfdb615 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.495710 29017 rpcz_store.cc:276] 0618 13:59:41.778463 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:45.161802 (+3383339us) service_pool.cc:224] Handling call
0618 13:59:45.495580 (+333778us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:45.481116 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:45.481255 (+ 139us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:45.481263 (+ 8us) write_op.cc:435] Acquired schema lock
0618 13:59:45.481266 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:45.481407 (+ 141us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:45.481422 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:45.481428 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:45.481530 (+ 102us) tablet.cc:701] Row locks acquired
0618 13:59:45.481533 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:45.481580 (+ 47us) write_op.cc:270] Start()
0618 13:59:45.481621 (+ 41us) write_op.cc:276] Timestamp: P: 1781791185481546 usec, L: 0
0618 13:59:45.481625 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:45.481822 (+ 197us) log.cc:844] Serialized 625 byte log entry
0618 13:59:45.487416 (+ 5594us) op_driver.cc:464] REPLICATION: finished
0618 13:59:45.487670 (+ 254us) write_op.cc:301] APPLY: starting
0618 13:59:45.487685 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:45.487731 (+ 46us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:45.487764 (+ 33us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:45.487870 (+ 106us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:45.487933 (+ 63us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:45.487940 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:45.488015 (+ 75us) log.cc:844] Serialized 29 byte log entry
0618 13:59:45.488082 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.488101 (+ 19us) write_op.cc:454] Released schema lock
0618 13:59:45.488108 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:45.488180 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.488184 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5406,"child_traces":[["op",{"apply.queue_time_us":200,"apply.run_cpu_time_us":502,"apply.run_wall_time_us":502,"num_ops":2,"prepare.queue_time_us":898,"prepare.run_cpu_time_us":826,"prepare.run_wall_time_us":827,"raft.queue_time_us":2219,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":310,"replication_time_us":5756}]]}
I20260618 13:59:45.497879 30950 ts_tablet_manager.cc:1916] T 6c4a31a4f202473d97b17b5b5dfdb615 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.498649 28543 raft_consensus.cc:3060] T 8af2bec820b842b4bf6b57482007bd6a P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.502853 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet 60b6216474b74a14af6abf977e312b77 with delete_type TABLET_DATA_DELETED (replaced by tablet f683917aac5945ec939664ab82c24530) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:45.503099 28645 tablet_service.cc:1558] Processing DeleteTablet for tablet 60b6216474b74a14af6abf977e312b77 with delete_type TABLET_DATA_DELETED (replaced by tablet f683917aac5945ec939664ab82c24530) from {username='slave'} at 127.0.0.1:43578
W20260618 13:59:45.502400 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet ba2d6894bf5649fc8cfae7e033c432b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba2d6894bf5649fc8cfae7e033c432b9 already in progress: creating tablet
I20260618 13:59:45.471560 28798 tablet_bootstrap.cc:492] T d8a71e334ed1400d97c965b49b677a81 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:45.509889 28798 tablet_bootstrap.cc:654] T d8a71e334ed1400d97c965b49b677a81 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.517688 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 60b6216474b74a14af6abf977e312b77 with delete_type TABLET_DATA_DELETED (replaced by tablet f683917aac5945ec939664ab82c24530) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:45.517925 30939 tablet_replica.cc:333] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:45.518160 30939 raft_consensus.cc:2243] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:45.518371 30939 raft_consensus.cc:2272] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.518478 30950 ts_tablet_manager.cc:1929] T 6c4a31a4f202473d97b17b5b5dfdb615 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:45.518983 30939 ts_tablet_manager.cc:1916] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.528880 30572 consensus_queue.cc:1048] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:45.529588 30986 tablet.cc:1724] T 8af2bec820b842b4bf6b57482007bd6a P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:45.536197 30950 log.cc:1199] T 6c4a31a4f202473d97b17b5b5dfdb615 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/6c4a31a4f202473d97b17b5b5dfdb615
I20260618 13:59:45.536629 30950 ts_tablet_manager.cc:1950] T 6c4a31a4f202473d97b17b5b5dfdb615 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:45.537528 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:45.538638 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 6c4a31a4f202473d97b17b5b5dfdb615 (table default.Table_152a483501b54a67bbdc0cae4959d452 [id=daf2b982b88e4021b327a0e52cf5316c]) successfully deleted
W20260618 13:59:45.543795 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 72c04f55890648f0a8184f3783243f7c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72c04f55890648f0a8184f3783243f7c already in progress: creating tablet
I20260618 13:59:45.544360 30950 tablet_replica.cc:333] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:45.544942 30950 raft_consensus.cc:2243] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:45.545439 30950 raft_consensus.cc:2272] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.545709 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:45.546797 30950 ts_tablet_manager.cc:1916] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.547683 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet e196dd3c7fe14524a50152643ddbc2ad (table default.Table_2dfebd62a146446f999f1a7a1bd1af79 [id=02ffed3d852845d6845fb5176f036fce]). Expected version 1 got 0
W20260618 13:59:45.553982 28618 consensus_peers.cc:597] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:45.556027 28618 consensus_peers.cc:597] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:45.557075 28798 tablet_bootstrap.cc:492] T d8a71e334ed1400d97c965b49b677a81 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:45.557207 28798 ts_tablet_manager.cc:1403] T d8a71e334ed1400d97c965b49b677a81 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.086s user 0.005s sys 0.000s
I20260618 13:59:45.557859 28798 raft_consensus.cc:359] T d8a71e334ed1400d97c965b49b677a81 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.558017 28798 raft_consensus.cc:385] T d8a71e334ed1400d97c965b49b677a81 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.558068 28798 raft_consensus.cc:740] T d8a71e334ed1400d97c965b49b677a81 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.558933 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:45.559296 30847 raft_consensus.cc:493] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.559499 30847 raft_consensus.cc:515] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.560693 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2d6894bf5649fc8cfae7e033c432b9" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:45.561262 28618 leader_election.cc:343] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.562192 30847 leader_election.cc:290] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:45.562541 28617 leader_election.cc:343] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: ba2d6894bf5649fc8cfae7e033c432b9
I20260618 13:59:45.562645 28617 leader_election.cc:304] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.562952 30847 raft_consensus.cc:2749] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:45.564195 28798 consensus_queue.cc:260] T d8a71e334ed1400d97c965b49b677a81 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.565210 28798 ts_tablet_manager.cc:1434] T d8a71e334ed1400d97c965b49b677a81 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260618 13:59:45.566066 28798 tablet_bootstrap.cc:492] T a8da96fc621c47d4a444b5a39259f85d P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:45.566740 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:45.568673 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 10e9a417eecf4d33ac3bd9bd6325f447 (table default.Table_21426dbf385d47d2af5fc25e287ed0ed [id=0de5461053eb4517b15d6b91875fc04d]). Expected version 1 got 0
I20260618 13:59:45.569037 28798 tablet_bootstrap.cc:654] T a8da96fc621c47d4a444b5a39259f85d P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.570555 28797 tablet_bootstrap.cc:492] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:45.570693 28797 ts_tablet_manager.cc:1403] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.078s user 0.004s sys 0.006s
I20260618 13:59:45.571457 28797 raft_consensus.cc:359] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.571623 28797 raft_consensus.cc:385] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.571683 28797 raft_consensus.cc:740] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.571748 30986 tablet.cc:1724] T 3f5d4296952e48a48756e6f27140da87 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:45.571959 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:45.571918 28797 consensus_queue.cc:260] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.572490 28797 ts_tablet_manager.cc:1434] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:45.572762 28797 tablet_bootstrap.cc:492] T 60e02f51b8624a739fadbf4fafe90cb5 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:45.574105 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:45.575007 28797 tablet_bootstrap.cc:654] T 60e02f51b8624a739fadbf4fafe90cb5 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.577116 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:45.577435 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:45.578635 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 00b602a473a94c05a617fb9478300c74 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
W20260618 13:59:45.579795 28983 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.100.60:42083: connect: Connection refused (error 111) [suppressed 10 similar messages]
I20260618 13:59:45.581218 28798 tablet_bootstrap.cc:492] T a8da96fc621c47d4a444b5a39259f85d P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:45.581341 28798 ts_tablet_manager.cc:1403] T a8da96fc621c47d4a444b5a39259f85d P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20260618 13:59:45.581336 31010 tablet.cc:1724] T 3f5d4296952e48a48756e6f27140da87 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:45.581964 28798 raft_consensus.cc:359] T a8da96fc621c47d4a444b5a39259f85d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.582170 28798 raft_consensus.cc:385] T a8da96fc621c47d4a444b5a39259f85d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.582245 28798 raft_consensus.cc:740] T a8da96fc621c47d4a444b5a39259f85d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.585587 31007 tablet.cc:1724] T 3f5d4296952e48a48756e6f27140da87 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:45.588793 30847 consensus_queue.cc:1048] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:45.590678 30847 raft_consensus.cc:493] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.590806 30847 raft_consensus.cc:515] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.591723 30847 leader_election.cc:290] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.592875 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba18e879631947f88bd266681746ec82" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:45.593362 28618 leader_election.cc:343] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.593668 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba18e879631947f88bd266681746ec82" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:45.593853 28407 raft_consensus.cc:2468] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:45.596422 28543 raft_consensus.cc:3060] T 5bbb71449d434d54b007d6f0da0796d7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.598560 28798 consensus_queue.cc:260] T a8da96fc621c47d4a444b5a39259f85d P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.598650 28617 leader_election.cc:304] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.599221 30950 ts_tablet_manager.cc:1929] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:45.599344 30854 raft_consensus.cc:2804] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:45.599434 30854 raft_consensus.cc:493] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:45.599505 30854 raft_consensus.cc:3060] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.600301 30950 log.cc:1199] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/962bf304a285412fb2a529db5b994e9e
I20260618 13:59:45.602243 30854 raft_consensus.cc:515] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.602773 30950 ts_tablet_manager.cc:1950] T 962bf304a285412fb2a529db5b994e9e P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:45.603808 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 962bf304a285412fb2a529db5b994e9e (table default.Table_0fbace1c1aa746a6a2a57c878631218b [id=18b5971926ed4fff9048003a51d3a4fa]) successfully deleted
I20260618 13:59:45.603981 28539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba18e879631947f88bd266681746ec82" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:45.604821 30950 tablet_replica.cc:333] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:45.605057 30950 raft_consensus.cc:2243] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:45.605347 30950 raft_consensus.cc:2272] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.605552 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba18e879631947f88bd266681746ec82" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:45.605716 28402 raft_consensus.cc:3060] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:45.605700 28618 leader_election.cc:343] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.605823 30854 leader_election.cc:290] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.605919 30950 ts_tablet_manager.cc:1916] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.608144 28402 raft_consensus.cc:2468] T ba18e879631947f88bd266681746ec82 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:45.608516 28617 leader_election.cc:304] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.600484 28798 ts_tablet_manager.cc:1434] T a8da96fc621c47d4a444b5a39259f85d P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.019s user 0.002s sys 0.000s
I20260618 13:59:45.609133 30898 raft_consensus.cc:2804] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:45.609287 30898 raft_consensus.cc:697] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:45.609562 28798 tablet_bootstrap.cc:492] T c25fa765ec794c34b0aa43761961a8c9 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:45.609602 30898 consensus_queue.cc:237] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.612360 29566 ts_tablet_manager.cc:1929] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:45.612501 29566 log.cc:1199] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/fea86899d2c047d997cebe80551e83c9
I20260618 13:59:45.612852 29566 ts_tablet_manager.cc:1950] T fea86899d2c047d997cebe80551e83c9 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:45.616129 28798 tablet_bootstrap.cc:654] T c25fa765ec794c34b0aa43761961a8c9 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.620218 30939 ts_tablet_manager.cc:1929] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:45.622732 30950 ts_tablet_manager.cc:1929] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:45.623638 30950 log.cc:1199] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/f45556a59c424ca2bc30956566374c89
I20260618 13:59:45.623953 30890 raft_consensus.cc:493] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.624109 30890 raft_consensus.cc:515] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.624413 30950 ts_tablet_manager.cc:1950] T f45556a59c424ca2bc30956566374c89 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:45.624802 30890 leader_election.cc:290] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.625926 28539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e438b3733c374e1bb8aae4ed9f16b4a4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:45.626012 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet f45556a59c424ca2bc30956566374c89 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]) successfully deleted
W20260618 13:59:45.626610 28618 leader_election.cc:343] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:45.627123 28617 consensus_peers.cc:597] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c9ceb3af08fa4618b81cf6f89d7bd0df. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:45.629491 30950 tablet_replica.cc:333] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:45.629909 30950 raft_consensus.cc:2243] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:45.630302 30950 raft_consensus.cc:2272] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.631009 30950 ts_tablet_manager.cc:1916] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.625711 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e438b3733c374e1bb8aae4ed9f16b4a4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:45.633612 28402 raft_consensus.cc:2468] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:45.633823 30939 log.cc:1199] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/60b6216474b74a14af6abf977e312b77
I20260618 13:59:45.634153 30939 ts_tablet_manager.cc:1950] T 60b6216474b74a14af6abf977e312b77 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:45.634114 28617 leader_election.cc:304] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.634948 30890 raft_consensus.cc:2804] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:45.635015 30890 raft_consensus.cc:493] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:45.635066 30890 raft_consensus.cc:3060] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.637845 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 372) took 2574 ms. Trace:
I20260618 13:59:45.638025 30552 raft_consensus.cc:493] T a662bb37fb7d4df49e61e524808b67a8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.637928 29566 rpcz_store.cc:276] 0618 13:59:43.063343 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:43.064080 (+ 737us) service_pool.cc:224] Handling call
0618 13:59:45.455805 (+2391725us) ts_tablet_manager.cc:1151] Deleting tablet fea86899d2c047d997cebe80551e83c9
0618 13:59:45.455816 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.558790 (+102974us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:45.612318 (+ 53528us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:45.637827 (+ 25509us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2391547}
I20260618 13:59:45.638132 30552 raft_consensus.cc:515] T a662bb37fb7d4df49e61e524808b67a8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.638751 30552 leader_election.cc:290] T a662bb37fb7d4df49e61e524808b67a8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.639716 28402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a662bb37fb7d4df49e61e524808b67a8" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:45.638360 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 373) took 2574 ms. Trace:
I20260618 13:59:45.640578 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a662bb37fb7d4df49e61e524808b67a8" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:45.640421 29566 rpcz_store.cc:276] 0618 13:59:43.063468 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:43.064276 (+ 808us) service_pool.cc:224] Handling call
0618 13:59:45.638201 (+2573925us) ts_tablet_manager.cc:1151] Deleting tablet 2128dbec33db453ba51070767a15b21e
0618 13:59:45.638211 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.638315 (+ 104us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2573800}
I20260618 13:59:45.642007 29566 tablet_replica.cc:333] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:45.642263 29566 raft_consensus.cc:2243] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:45.642827 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet fea86899d2c047d997cebe80551e83c9 (table default.Table_539ac216b6f4478eb994218d7d7980a4 [id=baa3214376034e9cae926def4c46b91e]) successfully deleted
I20260618 13:59:45.638752 30856 raft_consensus.cc:493] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260618 13:59:45.643115 28989 catalog_manager.cc:5007] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 2128dbec33db453ba51070767a15b21e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2128dbec33db453ba51070767a15b21e already in progress: creating tablet
I20260618 13:59:45.643090 30856 raft_consensus.cc:515] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.643579 30856 leader_election.cc:290] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.643702 28484 leader_election.cc:304] T a662bb37fb7d4df49e61e524808b67a8 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:45.644160 29566 raft_consensus.cc:2272] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.644325 30974 raft_consensus.cc:2749] T a662bb37fb7d4df49e61e524808b67a8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:45.644359 30890 raft_consensus.cc:515] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.644884 29566 ts_tablet_manager.cc:1916] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.645057 30890 leader_election.cc:290] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.646178 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e438b3733c374e1bb8aae4ed9f16b4a4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:45.646348 28408 raft_consensus.cc:3060] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.646368 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]) successfully deleted
I20260618 13:59:45.649102 28408 raft_consensus.cc:2468] T e438b3733c374e1bb8aae4ed9f16b4a4 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:45.646101 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f2d02ec004b9298b94307c7a38a97" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:45.649554 28407 raft_consensus.cc:2468] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:45.649886 28617 leader_election.cc:304] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:45.650092 30856 raft_consensus.cc:2804] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:45.650187 30856 raft_consensus.cc:697] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:45.650377 30856 consensus_queue.cc:237] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.652513 28617 leader_election.cc:304] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:45.653151 30856 raft_consensus.cc:2804] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:45.653234 30856 raft_consensus.cc:493] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:45.653311 30856 raft_consensus.cc:3060] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.655822 30856 raft_consensus.cc:515] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.656584 29566 ts_tablet_manager.cc:1929] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:45.656682 29566 log.cc:1199] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/c5d046e0421142cd84c5b12d6a434f53
I20260618 13:59:45.656977 30856 leader_election.cc:290] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.657258 29566 ts_tablet_manager.cc:1950] T c5d046e0421142cd84c5b12d6a434f53 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:45.658238 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f2d02ec004b9298b94307c7a38a97" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:45.658383 28408 raft_consensus.cc:3060] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.660761 28408 raft_consensus.cc:2468] T cb7f2d02ec004b9298b94307c7a38a97 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:45.661202 28617 leader_election.cc:304] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:45.661636 30890 raft_consensus.cc:2804] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:45.665957 30890 raft_consensus.cc:697] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:45.666129 29566 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43578 (request call id 375) took 1860 ms. Trace:
I20260618 13:59:45.666246 29566 rpcz_store.cc:276] 0618 13:59:43.805187 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:43.805322 (+ 135us) service_pool.cc:224] Handling call
0618 13:59:45.641937 (+1836615us) ts_tablet_manager.cc:1151] Deleting tablet c5d046e0421142cd84c5b12d6a434f53
0618 13:59:45.641946 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0618 13:59:45.653130 (+ 11184us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:45.656540 (+ 3410us) tablet_metadata.cc:665] Metadata flushed
0618 13:59:45.666109 (+ 9569us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1836437}
I20260618 13:59:45.666499 30890 consensus_queue.cc:237] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.666846 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet c5d046e0421142cd84c5b12d6a434f53 (table default.Table_689e7dd93b784dc5bbded16bd6cba279 [id=f6b4b4191cc74fe497234e6d2a0a071a]) successfully deleted
W20260618 13:59:45.667765 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet e5f7eaa7d218440ca2c35340c5c87814 because the tablet was not found. No further retry: Not found: Tablet not found: e5f7eaa7d218440ca2c35340c5c87814
W20260618 13:59:45.670003 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 9c17e76ae9b44207998a43e3576908c4 because the tablet was not found. No further retry: Not found: Tablet not found: 9c17e76ae9b44207998a43e3576908c4
W20260618 13:59:45.670663 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 283523e5b22843ba935eabe6283d3a99 because the tablet was not found. No further retry: Not found: Tablet not found: 283523e5b22843ba935eabe6283d3a99
I20260618 13:59:45.672588 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_01e1e5741bb54f78996857f6b79fc5d7 [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 2 got 0
I20260618 13:59:45.673234 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:45.674284 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:45.677639 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f2d02ec004b9298b94307c7a38a97" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:45.680354 29566 tablet_replica.cc:333] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:45.680640 29566 raft_consensus.cc:2243] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:45.680830 29566 raft_consensus.cc:2272] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.681507 29566 ts_tablet_manager.cc:1916] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.682505 28797 tablet_bootstrap.cc:492] T 60e02f51b8624a739fadbf4fafe90cb5 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:45.682664 28797 ts_tablet_manager.cc:1403] T 60e02f51b8624a739fadbf4fafe90cb5 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.110s user 0.011s sys 0.017s
I20260618 13:59:45.683924 28797 raft_consensus.cc:359] T 60e02f51b8624a739fadbf4fafe90cb5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.684116 28797 raft_consensus.cc:385] T 60e02f51b8624a739fadbf4fafe90cb5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.684165 28797 raft_consensus.cc:740] T 60e02f51b8624a739fadbf4fafe90cb5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.684369 28797 consensus_queue.cc:260] T 60e02f51b8624a739fadbf4fafe90cb5 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.684895 28797 ts_tablet_manager.cc:1434] T 60e02f51b8624a739fadbf4fafe90cb5 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:45.685245 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e438b3733c374e1bb8aae4ed9f16b4a4" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:45.685405 28797 tablet_bootstrap.cc:492] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:45.687505 28797 tablet_bootstrap.cc:654] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.687546 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f2d02ec004b9298b94307c7a38a97" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:45.688783 30950 ts_tablet_manager.cc:1929] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:45.690691 30950 log.cc:1199] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/d4667fe48f0f4eff9c6c8baabdab58d4
I20260618 13:59:45.691176 30950 ts_tablet_manager.cc:1950] T d4667fe48f0f4eff9c6c8baabdab58d4 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
W20260618 13:59:45.676318 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 46bfe8cc2078414d84e65e22ed4ef8a2 because the tablet was not found. No further retry: Not found: Tablet not found: 46bfe8cc2078414d84e65e22ed4ef8a2
W20260618 13:59:45.692493 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 962bf304a285412fb2a529db5b994e9e because the tablet was not found. No further retry: Not found: Tablet not found: 962bf304a285412fb2a529db5b994e9e
W20260618 13:59:45.692790 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 04dac931e62a448d99633878eb70b92f because the tablet was not found. No further retry: Not found: Tablet not found: 04dac931e62a448d99633878eb70b92f
W20260618 13:59:45.692991 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d4667fe48f0f4eff9c6c8baabdab58d4 because the tablet was not found. No further retry: Not found: Tablet not found: d4667fe48f0f4eff9c6c8baabdab58d4
W20260618 13:59:45.693228 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 55ae29c3b7ed44cfa659130745037aa6 because the tablet was not found. No further retry: Not found: Tablet not found: 55ae29c3b7ed44cfa659130745037aa6
W20260618 13:59:45.693436 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 9a0691de253b4b008e5d995fd6b876c2 because the tablet was not found. No further retry: Not found: Tablet not found: 9a0691de253b4b008e5d995fd6b876c2
W20260618 13:59:45.693643 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 5e25027f423144b088dad955c7282921 because the tablet was not found. No further retry: Not found: Tablet not found: 5e25027f423144b088dad955c7282921
W20260618 13:59:45.693830 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 6c4a31a4f202473d97b17b5b5dfdb615 because the tablet was not found. No further retry: Not found: Tablet not found: 6c4a31a4f202473d97b17b5b5dfdb615
W20260618 13:59:45.694011 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet f45556a59c424ca2bc30956566374c89 because the tablet was not found. No further retry: Not found: Tablet not found: f45556a59c424ca2bc30956566374c89
W20260618 13:59:45.694195 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet cd06bb7cffc34412b420d70435fce373 because the tablet was not found. No further retry: Not found: Tablet not found: cd06bb7cffc34412b420d70435fce373
W20260618 13:59:45.694383 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 35d8dd2c388e43a0be380aa953ec2c82 because the tablet was not found. No further retry: Not found: Tablet not found: 35d8dd2c388e43a0be380aa953ec2c82
W20260618 13:59:45.694598 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet d3dbf13544ce408ab97834aec5862390 because the tablet was not found. No further retry: Not found: Tablet not found: d3dbf13544ce408ab97834aec5862390
W20260618 13:59:45.694792 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 98b42d23832545ae81a7cdff19e4689f because the tablet was not found. No further retry: Not found: Tablet not found: 98b42d23832545ae81a7cdff19e4689f
W20260618 13:59:45.694962 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 97a9f949999446f09359629fe483bf67 because the tablet was not found. No further retry: Not found: Tablet not found: 97a9f949999446f09359629fe483bf67
W20260618 13:59:45.695160 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 3dd0eaa64271489d92aa380df877f257 because the tablet was not found. No further retry: Not found: Tablet not found: 3dd0eaa64271489d92aa380df877f257
W20260618 13:59:45.694388 28618 leader_election.cc:343] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:45.695350 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet ca2ee746c5024cfc9b31c470f9616e7d because the tablet was not found. No further retry: Not found: Tablet not found: ca2ee746c5024cfc9b31c470f9616e7d
W20260618 13:59:45.695539 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet c7f42997416841a7b2c1ad347d2e9282 because the tablet was not found. No further retry: Not found: Tablet not found: c7f42997416841a7b2c1ad347d2e9282
W20260618 13:59:45.695727 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet c4cfca9e76e641a7bc0b58b9ee240f36 because the tablet was not found. No further retry: Not found: Tablet not found: c4cfca9e76e641a7bc0b58b9ee240f36
W20260618 13:59:45.696012 28618 leader_election.cc:343] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.696593 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 8af2bec820b842b4bf6b57482007bd6a (table default.Table_84be6926f4b64876b343577b85db3243 [id=af1859c644b44c1d831314d27a494f94]). Expected version 1 got 0
I20260618 13:59:45.697494 30898 consensus_queue.cc:1048] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:45.697758 28618 leader_election.cc:343] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:45.698030 28618 consensus_peers.cc:597] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:45.698836 30950 tablet_replica.cc:333] T f2b21dcf74fa4264947ca43cc0026e76 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:45.702325 30950 raft_consensus.cc:2243] T f2b21dcf74fa4264947ca43cc0026e76 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:45.698578 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet d4667fe48f0f4eff9c6c8baabdab58d4 (table default.Table_0a85d79af65f48a3803bfd1248593a11 [id=2f4d62be772a421ab777fd23be58b719]) successfully deleted
I20260618 13:59:45.702792 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:45.703008 30950 raft_consensus.cc:2272] T f2b21dcf74fa4264947ca43cc0026e76 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.703055 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:45.703604 29023 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:45.703728 30950 ts_tablet_manager.cc:1916] T f2b21dcf74fa4264947ca43cc0026e76 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.708156 28798 tablet_bootstrap.cc:492] T c25fa765ec794c34b0aa43761961a8c9 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:45.708381 28798 ts_tablet_manager.cc:1403] T c25fa765ec794c34b0aa43761961a8c9 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.099s user 0.005s sys 0.009s
I20260618 13:59:45.709563 28798 raft_consensus.cc:359] T c25fa765ec794c34b0aa43761961a8c9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.709836 28798 raft_consensus.cc:385] T c25fa765ec794c34b0aa43761961a8c9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.709949 28798 raft_consensus.cc:740] T c25fa765ec794c34b0aa43761961a8c9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.710299 28798 consensus_queue.cc:260] T c25fa765ec794c34b0aa43761961a8c9 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.710628 30950 ts_tablet_manager.cc:1929] T f2b21dcf74fa4264947ca43cc0026e76 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:45.710793 30950 log.cc:1199] T f2b21dcf74fa4264947ca43cc0026e76 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/f2b21dcf74fa4264947ca43cc0026e76
I20260618 13:59:45.711042 28798 ts_tablet_manager.cc:1434] T c25fa765ec794c34b0aa43761961a8c9 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260618 13:59:45.711385 28798 tablet_bootstrap.cc:492] T 8bef52c0fc354d6cb5388700641af9f8 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:45.712214 30950 ts_tablet_manager.cc:1950] T f2b21dcf74fa4264947ca43cc0026e76 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:45.712971 28798 tablet_bootstrap.cc:654] T 8bef52c0fc354d6cb5388700641af9f8 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.716464 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet f2b21dcf74fa4264947ca43cc0026e76 (table default.Table_c35470a550534fa2ad072f932e981df5 [id=2797c7fefad74c1e82ab31fb9ad4db70]) successfully deleted
I20260618 13:59:45.718395 29566 ts_tablet_manager.cc:1929] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:45.718888 29566 log.cc:1199] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/e3d8616846c649b6a969b971db0d03db
I20260618 13:59:45.719307 29566 ts_tablet_manager.cc:1950] T e3d8616846c649b6a969b971db0d03db P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:45.721890 30572 consensus_queue.cc:1048] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:45.726893 28617 consensus_peers.cc:597] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Couldn't send request to peer 6fc808af0e884ed8836561665d8e0e7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60b6216474b74a14af6abf977e312b77. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:45.731317 28618 consensus_peers.cc:597] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:45.733294 28798 tablet_bootstrap.cc:492] T 8bef52c0fc354d6cb5388700641af9f8 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:45.733454 28798 ts_tablet_manager.cc:1403] T 8bef52c0fc354d6cb5388700641af9f8 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.002s
I20260618 13:59:45.734982 28798 raft_consensus.cc:359] T 8bef52c0fc354d6cb5388700641af9f8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.735311 28798 raft_consensus.cc:385] T 8bef52c0fc354d6cb5388700641af9f8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.735415 28798 raft_consensus.cc:740] T 8bef52c0fc354d6cb5388700641af9f8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.735550 28797 tablet_bootstrap.cc:492] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:45.735687 28797 ts_tablet_manager.cc:1403] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.051s user 0.003s sys 0.000s
I20260618 13:59:45.735834 28798 consensus_queue.cc:260] T 8bef52c0fc354d6cb5388700641af9f8 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.736459 28797 raft_consensus.cc:359] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.736239 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet e3d8616846c649b6a969b971db0d03db (table default.Table_473a0c23203b404ea8e5ee0aed201e7b [id=3656a2f8d6cc4892a071908ae55efd3f]) successfully deleted
I20260618 13:59:45.736645 28797 raft_consensus.cc:385] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.736706 28797 raft_consensus.cc:740] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
W20260618 13:59:45.736846 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet f278d0acbb0c4a0ca61640bc142e6335 because the tablet was not found. No further retry: Not found: Tablet not found: f278d0acbb0c4a0ca61640bc142e6335
I20260618 13:59:45.736886 28797 consensus_queue.cc:260] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:45.737082 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet e6de1293572f4e75af3417f281b9a142 because the tablet was not found. No further retry: Not found: Tablet not found: e6de1293572f4e75af3417f281b9a142
I20260618 13:59:45.737332 28797 ts_tablet_manager.cc:1434] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260618 13:59:45.737277 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 827cc0987bd541759e334fb620f04ed5 because the tablet was not found. No further retry: Not found: Tablet not found: 827cc0987bd541759e334fb620f04ed5
W20260618 13:59:45.737681 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet be59f988f0e34b9480f859cf88b9c223 because the tablet was not found. No further retry: Not found: Tablet not found: be59f988f0e34b9480f859cf88b9c223
I20260618 13:59:45.737702 28797 tablet_bootstrap.cc:492] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:45.737867 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 015494374645469785f1f978eaf0e9a5 because the tablet was not found. No further retry: Not found: Tablet not found: 015494374645469785f1f978eaf0e9a5
W20260618 13:59:45.738102 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 because the tablet was not found. No further retry: Not found: Tablet not found: 75b9c0754beb4e0e8ed3a7c1e9f368a9
W20260618 13:59:45.738310 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet ac58e939730c4af681c2f08bcfbb6602 because the tablet was not found. No further retry: Not found: Tablet not found: ac58e939730c4af681c2f08bcfbb6602
W20260618 13:59:45.738535 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet cea4bc13eabc4dd98b4f96723498014c because the tablet was not found. No further retry: Not found: Tablet not found: cea4bc13eabc4dd98b4f96723498014c
W20260618 13:59:45.738737 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 864f2602840c40fc8606b3784e671cba because the tablet was not found. No further retry: Not found: Tablet not found: 864f2602840c40fc8606b3784e671cba
W20260618 13:59:45.738973 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 591465e2b57247a2b224cc8b09225811 because the tablet was not found. No further retry: Not found: Tablet not found: 591465e2b57247a2b224cc8b09225811
I20260618 13:59:45.739655 28797 tablet_bootstrap.cc:654] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.742093 28798 ts_tablet_manager.cc:1434] T 8bef52c0fc354d6cb5388700641af9f8 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260618 13:59:45.742432 28798 tablet_bootstrap.cc:492] T ff8e487fd51f4aa18bf59e5f8e011dcd P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
W20260618 13:59:45.744613 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet c3d523419ff244f085d6eac1a3231973 because the tablet was not found. No further retry: Not found: Tablet not found: c3d523419ff244f085d6eac1a3231973
I20260618 13:59:45.744880 29566 tablet_replica.cc:333] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:45.745074 29566 raft_consensus.cc:2243] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:45.745211 29566 raft_consensus.cc:2272] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.745707 29566 ts_tablet_manager.cc:1916] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:45.747792 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet f3ba9186b3ff47d5a2b158fef619686e because the tablet was not found. No further retry: Not found: Tablet not found: f3ba9186b3ff47d5a2b158fef619686e
W20260618 13:59:45.748013 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 98446f40214c4ac984fcfe98310005c5 because the tablet was not found. No further retry: Not found: Tablet not found: 98446f40214c4ac984fcfe98310005c5
W20260618 13:59:45.748227 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet c01c8463055e4526b3c75190066f30b6 because the tablet was not found. No further retry: Not found: Tablet not found: c01c8463055e4526b3c75190066f30b6
W20260618 13:59:45.748392 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 7572d17724d249739be1a1cb9468f909 because the tablet was not found. No further retry: Not found: Tablet not found: 7572d17724d249739be1a1cb9468f909
W20260618 13:59:45.748680 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet ab521028ac1a4d0da013112d76bb169f because the tablet was not found. No further retry: Not found: Tablet not found: ab521028ac1a4d0da013112d76bb169f
W20260618 13:59:45.748898 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet 0fae928833e44d2ba860d6184d4c0142 because the tablet was not found. No further retry: Not found: Tablet not found: 0fae928833e44d2ba860d6184d4c0142
W20260618 13:59:45.749145 28989 catalog_manager.cc:4994] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): delete failed for tablet f2b21dcf74fa4264947ca43cc0026e76 because the tablet was not found. No further retry: Not found: Tablet not found: f2b21dcf74fa4264947ca43cc0026e76
I20260618 13:59:45.750342 28798 tablet_bootstrap.cc:654] T ff8e487fd51f4aa18bf59e5f8e011dcd P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.751813 29566 ts_tablet_manager.cc:1929] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:45.751919 29566 log.cc:1199] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/ba2d6894bf5649fc8cfae7e033c432b9
I20260618 13:59:45.752218 29566 ts_tablet_manager.cc:1950] T ba2d6894bf5649fc8cfae7e033c432b9 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
W20260618 13:59:45.752826 28346 consensus_peers.cc:597] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:45.753047 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 5e25027f423144b088dad955c7282921 because the tablet was not found. No further retry: Not found: Tablet not found: 5e25027f423144b088dad955c7282921
I20260618 13:59:45.753314 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet ba2d6894bf5649fc8cfae7e033c432b9 (table default.Table_dba55442642c45b1b8dfe959d62699c9 [id=596cac3f4f6540019b75bb586ec7f9ab]) successfully deleted
I20260618 13:59:45.753363 29566 tablet_replica.cc:333] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:45.753544 29566 raft_consensus.cc:2243] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:45.753849 29566 raft_consensus.cc:2272] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 13:59:45.753923 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 55ae29c3b7ed44cfa659130745037aa6 because the tablet was not found. No further retry: Not found: Tablet not found: 55ae29c3b7ed44cfa659130745037aa6
W20260618 13:59:45.754458 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet c3d523419ff244f085d6eac1a3231973 because the tablet was not found. No further retry: Not found: Tablet not found: c3d523419ff244f085d6eac1a3231973
I20260618 13:59:45.754530 29566 ts_tablet_manager.cc:1916] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.754752 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
W20260618 13:59:45.755173 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet f3ba9186b3ff47d5a2b158fef619686e because the tablet was not found. No further retry: Not found: Tablet not found: f3ba9186b3ff47d5a2b158fef619686e
W20260618 13:59:45.759155 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet c4cfca9e76e641a7bc0b58b9ee240f36 because the tablet was not found. No further retry: Not found: Tablet not found: c4cfca9e76e641a7bc0b58b9ee240f36
W20260618 13:59:45.759418 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 35d8dd2c388e43a0be380aa953ec2c82 because the tablet was not found. No further retry: Not found: Tablet not found: 35d8dd2c388e43a0be380aa953ec2c82
W20260618 13:59:45.759640 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 7aa9f65d8a5c46fdb745ee17af946dce because the tablet was not found. No further retry: Not found: Tablet not found: 7aa9f65d8a5c46fdb745ee17af946dce
W20260618 13:59:45.759829 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 18d8b609b96a4e81b57f69bff522eeea because the tablet was not found. No further retry: Not found: Tablet not found: 18d8b609b96a4e81b57f69bff522eeea
I20260618 13:59:45.759994 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:45.760113 28797 tablet_bootstrap.cc:492] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:45.760241 28797 ts_tablet_manager.cc:1403] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20260618 13:59:45.760880 28797 raft_consensus.cc:359] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.761047 28797 raft_consensus.cc:385] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.761124 28797 raft_consensus.cc:740] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.761322 28797 consensus_queue.cc:260] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.761664 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:45.762007 28797 ts_tablet_manager.cc:1434] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:45.762274 28797 tablet_bootstrap.cc:492] T 6903cb63c5b04408b7eaecf0c0231c42 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:45.762975 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:45.763314 29566 ts_tablet_manager.cc:1929] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:45.763419 29566 log.cc:1199] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/91d014b46a0f448b929cbc9287ead0a7
I20260618 13:59:45.763792 29566 ts_tablet_manager.cc:1950] T 91d014b46a0f448b929cbc9287ead0a7 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
W20260618 13:59:45.760088 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 2f3d2f236fd44ab8af0524ea282cfa94 because the tablet was not found. No further retry: Not found: Tablet not found: 2f3d2f236fd44ab8af0524ea282cfa94
I20260618 13:59:45.764219 28797 tablet_bootstrap.cc:654] T 6903cb63c5b04408b7eaecf0c0231c42 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:45.764326 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet c7f42997416841a7b2c1ad347d2e9282 because the tablet was not found. No further retry: Not found: Tablet not found: c7f42997416841a7b2c1ad347d2e9282
I20260618 13:59:45.764786 29566 tablet_replica.cc:333] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:45.764389 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:45.764972 29566 raft_consensus.cc:2243] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:45.765231 29566 pending_rounds.cc:70] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257: Trying to abort 1 pending ops.
I20260618 13:59:45.765297 29566 pending_rounds.cc:77] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298216691250028544 op_type: NO_OP noop_request { }
I20260618 13:59:45.765391 29566 raft_consensus.cc:2889] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:45.765463 29566 raft_consensus.cc:2272] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.765967 29566 ts_tablet_manager.cc:1916] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.765266 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:45.768303 30950 tablet_replica.cc:333] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:45.768543 30950 raft_consensus.cc:2243] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:45.768782 30950 raft_consensus.cc:2272] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.769323 30950 ts_tablet_manager.cc:1916] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:45.764549 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 125bb6885ab348d885d5a3b4bdd45bc7 because the tablet was not found. No further retry: Not found: Tablet not found: 125bb6885ab348d885d5a3b4bdd45bc7
I20260618 13:59:45.770673 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet e196dd3c7fe14524a50152643ddbc2ad (table default.Table_2dfebd62a146446f999f1a7a1bd1af79 [id=02ffed3d852845d6845fb5176f036fce]). Expected version 1 got 0
W20260618 13:59:45.770753 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet c552d1316e3340f4bcfa39618ae309b8 because the tablet was not found. No further retry: Not found: Tablet not found: c552d1316e3340f4bcfa39618ae309b8
I20260618 13:59:45.771015 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_01e1e5741bb54f78996857f6b79fc5d7 [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 2 got 0
W20260618 13:59:45.771199 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 7d4144b1b5ea4e968672616bc22bead8 because the tablet was not found. No further retry: Not found: Tablet not found: 7d4144b1b5ea4e968672616bc22bead8
I20260618 13:59:45.771338 28797 tablet_bootstrap.cc:492] T 6903cb63c5b04408b7eaecf0c0231c42 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:45.771453 28797 ts_tablet_manager.cc:1403] T 6903cb63c5b04408b7eaecf0c0231c42 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.003s
I20260618 13:59:45.772032 28797 raft_consensus.cc:359] T 6903cb63c5b04408b7eaecf0c0231c42 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.772496 28797 raft_consensus.cc:385] T 6903cb63c5b04408b7eaecf0c0231c42 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.772552 28797 raft_consensus.cc:740] T 6903cb63c5b04408b7eaecf0c0231c42 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.772732 28797 consensus_queue.cc:260] T 6903cb63c5b04408b7eaecf0c0231c42 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.773396 28797 ts_tablet_manager.cc:1434] T 6903cb63c5b04408b7eaecf0c0231c42 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 13:59:45.773641 28797 tablet_bootstrap.cc:492] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:45.775586 28797 tablet_bootstrap.cc:654] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:45.771946 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 827cc0987bd541759e334fb620f04ed5 because the tablet was not found. No further retry: Not found: Tablet not found: 827cc0987bd541759e334fb620f04ed5
W20260618 13:59:45.777789 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 864f2602840c40fc8606b3784e671cba because the tablet was not found. No further retry: Not found: Tablet not found: 864f2602840c40fc8606b3784e671cba
W20260618 13:59:45.777999 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 591465e2b57247a2b224cc8b09225811 because the tablet was not found. No further retry: Not found: Tablet not found: 591465e2b57247a2b224cc8b09225811
W20260618 13:59:45.778205 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet cd06bb7cffc34412b420d70435fce373 because the tablet was not found. No further retry: Not found: Tablet not found: cd06bb7cffc34412b420d70435fce373
I20260618 13:59:45.778330 28798 tablet_bootstrap.cc:492] T ff8e487fd51f4aa18bf59e5f8e011dcd P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
W20260618 13:59:45.778463 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 75b9c0754beb4e0e8ed3a7c1e9f368a9 because the tablet was not found. No further retry: Not found: Tablet not found: 75b9c0754beb4e0e8ed3a7c1e9f368a9
I20260618 13:59:45.778501 28798 ts_tablet_manager.cc:1403] T ff8e487fd51f4aa18bf59e5f8e011dcd P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.036s user 0.004s sys 0.000s
W20260618 13:59:45.778720 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet e6de1293572f4e75af3417f281b9a142 because the tablet was not found. No further retry: Not found: Tablet not found: e6de1293572f4e75af3417f281b9a142
I20260618 13:59:45.778915 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 91d014b46a0f448b929cbc9287ead0a7 (table default.Table_62bf5ce52fb14bd089678663dd009a91 [id=713af0f85a434ed0bfb791c48f75dc9d]) successfully deleted
W20260618 13:59:45.779122 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet f278d0acbb0c4a0ca61640bc142e6335 because the tablet was not found. No further retry: Not found: Tablet not found: f278d0acbb0c4a0ca61640bc142e6335
W20260618 13:59:45.779305 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 02b5cc08ca0c4e719e0414b345211ba5 because the tablet was not found. No further retry: Not found: Tablet not found: 02b5cc08ca0c4e719e0414b345211ba5
I20260618 13:59:45.779217 28798 raft_consensus.cc:359] T ff8e487fd51f4aa18bf59e5f8e011dcd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.779369 28798 raft_consensus.cc:385] T ff8e487fd51f4aa18bf59e5f8e011dcd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.779426 28798 raft_consensus.cc:740] T ff8e487fd51f4aa18bf59e5f8e011dcd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
W20260618 13:59:45.779487 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet cea4bc13eabc4dd98b4f96723498014c because the tablet was not found. No further retry: Not found: Tablet not found: cea4bc13eabc4dd98b4f96723498014c
I20260618 13:59:45.779531 30262 consensus_queue.cc:1048] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:45.779666 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 6eea4d59935d40aa97dc10ec7469aa56 because the tablet was not found. No further retry: Not found: Tablet not found: 6eea4d59935d40aa97dc10ec7469aa56
I20260618 13:59:45.779666 28798 consensus_queue.cc:260] T ff8e487fd51f4aa18bf59e5f8e011dcd P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:45.779850 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 0463fa2e30554abcbfc8f37b702e81d4 because the tablet was not found. No further retry: Not found: Tablet not found: 0463fa2e30554abcbfc8f37b702e81d4
W20260618 13:59:45.780210 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet be59f988f0e34b9480f859cf88b9c223 because the tablet was not found. No further retry: Not found: Tablet not found: be59f988f0e34b9480f859cf88b9c223
W20260618 13:59:45.780396 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet ca2ee746c5024cfc9b31c470f9616e7d because the tablet was not found. No further retry: Not found: Tablet not found: ca2ee746c5024cfc9b31c470f9616e7d
W20260618 13:59:45.780550 28989 catalog_manager.cc:4994] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 68b8293a7c924a449307ab88a1cf9290 because the tablet was not found. No further retry: Not found: Tablet not found: 68b8293a7c924a449307ab88a1cf9290
I20260618 13:59:45.771536 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:45.781116 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:45.783000 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 00b602a473a94c05a617fb9478300c74 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:45.784263 28798 ts_tablet_manager.cc:1434] T ff8e487fd51f4aa18bf59e5f8e011dcd P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260618 13:59:45.784548 28798 tablet_bootstrap.cc:492] T 2e493f647a344cd581515a77230d765f P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:45.787094 28798 tablet_bootstrap.cc:654] T 2e493f647a344cd581515a77230d765f P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.787763 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:45.792675 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:45.794060 30262 raft_consensus.cc:493] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.794023 29020 catalog_manager.cc:5696] T d3ed5c1137434fcc80ed0ca56a323be7 P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:45.794194 30262 raft_consensus.cc:515] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.794700 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:45.794795 30262 leader_election.cc:290] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:45.794991 28797 tablet_bootstrap.cc:492] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:45.795133 28797 ts_tablet_manager.cc:1403] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.003s
I20260618 13:59:45.795852 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:45.795823 28797 raft_consensus.cc:359] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.795993 28797 raft_consensus.cc:385] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.796054 28797 raft_consensus.cc:740] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.796250 28797 consensus_queue.cc:260] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.797164 28797 ts_tablet_manager.cc:1434] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 13:59:45.797425 28797 tablet_bootstrap.cc:492] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:45.798538 29566 ts_tablet_manager.cc:1929] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:45.798652 29566 log.cc:1199] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/c9ceb3af08fa4618b81cf6f89d7bd0df
I20260618 13:59:45.798620 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6903cb63c5b04408b7eaecf0c0231c42" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:45.799001 29566 ts_tablet_manager.cc:1950] T c9ceb3af08fa4618b81cf6f89d7bd0df P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:45.799217 28797 tablet_bootstrap.cc:654] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:45.799319 28346 leader_election.cc:343] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.800753 28798 tablet_bootstrap.cc:492] T 2e493f647a344cd581515a77230d765f P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:45.800917 28798 ts_tablet_manager.cc:1403] T 2e493f647a344cd581515a77230d765f P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.000s
I20260618 13:59:45.801851 28798 raft_consensus.cc:359] T 2e493f647a344cd581515a77230d765f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.801997 29566 tablet_replica.cc:333] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:45.802026 28798 raft_consensus.cc:385] T 2e493f647a344cd581515a77230d765f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.802101 28798 raft_consensus.cc:740] T 2e493f647a344cd581515a77230d765f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.802201 29566 raft_consensus.cc:2243] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:45.802503 29566 pending_rounds.cc:70] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257: Trying to abort 1 pending ops.
I20260618 13:59:45.802575 29566 pending_rounds.cc:77] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298216692038717440 op_type: NO_OP noop_request { }
I20260618 13:59:45.802666 29566 raft_consensus.cc:2889] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:45.802606 28798 consensus_queue.cc:260] T 2e493f647a344cd581515a77230d765f P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.802750 29566 raft_consensus.cc:2272] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:45.803216 28798 ts_tablet_manager.cc:1434] T 2e493f647a344cd581515a77230d765f P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:45.803249 29566 ts_tablet_manager.cc:1916] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:45.803484 28798 tablet_bootstrap.cc:492] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:45.803591 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6903cb63c5b04408b7eaecf0c0231c42" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:45.803782 28682 raft_consensus.cc:2468] T 6903cb63c5b04408b7eaecf0c0231c42 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:45.804944 28798 tablet_bootstrap.cc:654] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.805014 28543 raft_consensus.cc:3060] T 2e493f647a344cd581515a77230d765f P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.808836 28346 leader_election.cc:304] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.809374 30152 raft_consensus.cc:2804] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:45.809458 30152 raft_consensus.cc:493] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:45.809516 30152 raft_consensus.cc:3060] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.811136 31007 tablet.cc:1724] T 3a783590159f4a80964fbbcb4a24eef9 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:45.812500 30152 raft_consensus.cc:515] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.813105 30152 leader_election.cc:290] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:45.801406 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet c9ceb3af08fa4618b81cf6f89d7bd0df (table default.Table_21242eba3cb64c7c91082e296439cfe2 [id=10a4f5ef579746b68d8dd37d83f14c3e]) successfully deleted
I20260618 13:59:45.814703 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:45.814846 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6903cb63c5b04408b7eaecf0c0231c42" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:45.814960 29020 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 10e9a417eecf4d33ac3bd9bd6325f447 (table default.Table_21426dbf385d47d2af5fc25e287ed0ed [id=0de5461053eb4517b15d6b91875fc04d]). Expected version 1 got 0
I20260618 13:59:45.815382 29020 catalog_manager.cc:5696] T 0e6f3f8431cd49b5bf1306bc7609761d P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:45.815593 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6903cb63c5b04408b7eaecf0c0231c42" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:45.815738 28682 raft_consensus.cc:3060] T 6903cb63c5b04408b7eaecf0c0231c42 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.815855 31010 tablet.cc:1724] T 3a783590159f4a80964fbbcb4a24eef9 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:45.815915 29020 catalog_manager.cc:5696] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:45.818085 28682 raft_consensus.cc:2468] T 6903cb63c5b04408b7eaecf0c0231c42 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
W20260618 13:59:45.819413 28346 leader_election.cc:343] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.819624 28346 leader_election.cc:304] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.820096 30622 raft_consensus.cc:2804] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:45.820230 30622 raft_consensus.cc:697] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:45.820463 30622 consensus_queue.cc:237] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.823926 30986 tablet.cc:1724] T 3a783590159f4a80964fbbcb4a24eef9 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:45.824496 28596 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:45.827760 28798 tablet_bootstrap.cc:492] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:45.827903 28798 ts_tablet_manager.cc:1403] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.001s
I20260618 13:59:45.828753 28798 raft_consensus.cc:359] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.828953 28798 raft_consensus.cc:385] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.829011 28798 raft_consensus.cc:740] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.829193 28797 tablet_bootstrap.cc:492] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:45.829206 28798 consensus_queue.cc:260] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.829329 28797 ts_tablet_manager.cc:1403] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.000s
I20260618 13:59:45.829716 28798 ts_tablet_manager.cc:1434] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 13:59:45.830029 28797 raft_consensus.cc:359] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.830199 28797 raft_consensus.cc:385] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.830261 28797 raft_consensus.cc:740] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.830472 28797 consensus_queue.cc:260] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.831051 28797 ts_tablet_manager.cc:1434] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:45.831321 28797 tablet_bootstrap.cc:492] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:45.832973 28797 tablet_bootstrap.cc:654] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.835359 28798 tablet_bootstrap.cc:492] T 60b6216474b74a14af6abf977e312b77 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:45.837036 28539 raft_consensus.cc:3060] T 5115a31a38db48c1a6474917aff97c8e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.838475 30854 raft_consensus.cc:493] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.838596 30854 raft_consensus.cc:515] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.839120 30854 leader_election.cc:290] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.839382 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "732c8801c319400283806aac7f15917e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:45.839600 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "732c8801c319400283806aac7f15917e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:45.839766 28408 raft_consensus.cc:2468] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:45.839912 28618 leader_election.cc:343] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.840140 28617 leader_election.cc:304] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.840624 30854 raft_consensus.cc:2804] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:45.840708 30854 raft_consensus.cc:493] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:45.840785 30854 raft_consensus.cc:3060] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.843503 30854 raft_consensus.cc:515] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.843918 30572 consensus_queue.cc:1048] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:45.845325 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "732c8801c319400283806aac7f15917e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:45.845517 30854 leader_election.cc:290] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.845526 28797 tablet_bootstrap.cc:492] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:45.845707 28797 ts_tablet_manager.cc:1403] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.001s
W20260618 13:59:45.845806 28618 leader_election.cc:343] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.846083 29566 ts_tablet_manager.cc:1929] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:45.846184 29566 log.cc:1199] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/60b6216474b74a14af6abf977e312b77
I20260618 13:59:45.846482 28797 raft_consensus.cc:359] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.846700 28797 raft_consensus.cc:385] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.846755 28797 raft_consensus.cc:740] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:45.847371 28797 consensus_queue.cc:260] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.847765 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "732c8801c319400283806aac7f15917e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:45.847975 28797 ts_tablet_manager.cc:1434] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:45.848131 28411 raft_consensus.cc:3060] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.848237 28797 tablet_bootstrap.cc:492] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:45.849151 29566 ts_tablet_manager.cc:1950] T 60b6216474b74a14af6abf977e312b77 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:45.849586 28797 tablet_bootstrap.cc:654] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.850111 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 60b6216474b74a14af6abf977e312b77 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]) successfully deleted
I20260618 13:59:45.852097 28798 tablet_bootstrap.cc:654] T 60b6216474b74a14af6abf977e312b77 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.856950 30572 consensus_queue.cc:1048] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:45.871237 28797 tablet_bootstrap.cc:492] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:45.871368 28797 ts_tablet_manager.cc:1403] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
W20260618 13:59:45.871802 30634 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
I20260618 13:59:45.871991 28797 raft_consensus.cc:359] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.872148 28797 raft_consensus.cc:385] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:45.872200 28797 raft_consensus.cc:740] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
W20260618 13:59:45.872413 30634 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260618 13:59:45.872393 28797 consensus_queue.cc:260] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:45.872668 30634 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260618 13:59:45.872864 28797 ts_tablet_manager.cc:1434] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 13:59:45.873119 28797 tablet_bootstrap.cc:492] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:45.875094 28797 tablet_bootstrap.cc:654] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.876262 28411 raft_consensus.cc:2468] T 732c8801c319400283806aac7f15917e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:45.876884 28617 leader_election.cc:304] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.877338 30854 raft_consensus.cc:2804] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:45.878165 30854 raft_consensus.cc:697] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:45.878702 30854 consensus_queue.cc:237] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.895116 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 7c96a195e54440d4854e4462d178b468 (DEFAULT_TABLE table=default.Table_1aaf686e997e41f490f09c2e23259f74 [id=fa77b72087d54779b2330b84557c9af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.895555 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c96a195e54440d4854e4462d178b468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.896905 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 7c96a195e54440d4854e4462d178b468 (DEFAULT_TABLE table=default.Table_1aaf686e997e41f490f09c2e23259f74 [id=fa77b72087d54779b2330b84557c9af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.897364 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c96a195e54440d4854e4462d178b468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.899746 28651 tablet_service.cc:1511] Processing CreateTablet for tablet 7c96a195e54440d4854e4462d178b468 (DEFAULT_TABLE table=default.Table_1aaf686e997e41f490f09c2e23259f74 [id=fa77b72087d54779b2330b84557c9af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.900148 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c96a195e54440d4854e4462d178b468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.900372 30552 raft_consensus.cc:493] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.900490 30552 raft_consensus.cc:515] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.901154 30552 leader_election.cc:290] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.901798 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db62b3a1e37b447c9ea3fbd65c0cf1e7" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:45.903250 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db62b3a1e37b447c9ea3fbd65c0cf1e7" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:45.906860 28484 leader_election.cc:304] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:45.907523 30552 raft_consensus.cc:2749] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:45.908905 30634 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 13:59:45.913663 30634 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260618 13:59:45.913884 30634 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 13:59:45.914091 30634 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260618 13:59:45.915457 31048 tablet_bootstrap.cc:492] T 7c96a195e54440d4854e4462d178b468 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:45.920336 31048 tablet_bootstrap.cc:654] T 7c96a195e54440d4854e4462d178b468 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:45.925211 28526 tablet_service.cc:1511] Processing CreateTablet for tablet 55df96d305bf466b9fdea88a1816b2fa (DEFAULT_TABLE table=default.Table_e2020f3cb00e47e88950c4a8883e0ae4 [id=9c637e0b54e0405da391ce0e9412147c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.925778 28526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55df96d305bf466b9fdea88a1816b2fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.925899 30577 raft_consensus.cc:493] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.926045 30577 raft_consensus.cc:515] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.926726 30577 leader_election.cc:290] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.927464 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c200f34abf074684a0314c9024f04caf" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:45.928205 28618 leader_election.cc:343] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.930073 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c200f34abf074684a0314c9024f04caf" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:45.930279 28407 raft_consensus.cc:2468] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:45.932092 28617 leader_election.cc:304] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.932705 29303 raft_consensus.cc:2804] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:45.932809 29303 raft_consensus.cc:493] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:45.932883 29303 raft_consensus.cc:3060] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.934954 28383 tablet_service.cc:1511] Processing CreateTablet for tablet 55df96d305bf466b9fdea88a1816b2fa (DEFAULT_TABLE table=default.Table_e2020f3cb00e47e88950c4a8883e0ae4 [id=9c637e0b54e0405da391ce0e9412147c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.935520 28383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55df96d305bf466b9fdea88a1816b2fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.936220 29303 raft_consensus.cc:515] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.936475 28651 tablet_service.cc:1511] Processing CreateTablet for tablet 55df96d305bf466b9fdea88a1816b2fa (DEFAULT_TABLE table=default.Table_e2020f3cb00e47e88950c4a8883e0ae4 [id=9c637e0b54e0405da391ce0e9412147c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.936935 29303 leader_election.cc:290] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.937186 28548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c200f34abf074684a0314c9024f04caf" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:45.938357 28618 leader_election.cc:343] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.939209 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c200f34abf074684a0314c9024f04caf" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:45.939263 28517 tablet_service.cc:1511] Processing CreateTablet for tablet 5b51ac996e88453eb204e8b83f33863a (DEFAULT_TABLE table=default.Table_7438ba53b4e24b31a44bce5986fbf884 [id=278cfb83a6c9405cb2aa8cc23bf2398a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.939352 28403 raft_consensus.cc:3060] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.939613 28517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b51ac996e88453eb204e8b83f33863a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.940409 28528 tablet_service.cc:1511] Processing CreateTablet for tablet e772ef1424fc4a8f8d9e3288e718e21a (DEFAULT_TABLE table=default.Table_1b1680a2f241430aa5d5702997655d36 [id=65c0597739b84caeb72d9d4c8a0636a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.940718 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e772ef1424fc4a8f8d9e3288e718e21a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.942199 28377 tablet_service.cc:1511] Processing CreateTablet for tablet 5b51ac996e88453eb204e8b83f33863a (DEFAULT_TABLE table=default.Table_7438ba53b4e24b31a44bce5986fbf884 [id=278cfb83a6c9405cb2aa8cc23bf2398a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.942549 28377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b51ac996e88453eb204e8b83f33863a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.943183 28375 tablet_service.cc:1511] Processing CreateTablet for tablet e772ef1424fc4a8f8d9e3288e718e21a (DEFAULT_TABLE table=default.Table_1b1680a2f241430aa5d5702997655d36 [id=65c0597739b84caeb72d9d4c8a0636a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.943492 28375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e772ef1424fc4a8f8d9e3288e718e21a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.950227 28645 tablet_service.cc:1511] Processing CreateTablet for tablet 5b51ac996e88453eb204e8b83f33863a (DEFAULT_TABLE table=default.Table_7438ba53b4e24b31a44bce5986fbf884 [id=278cfb83a6c9405cb2aa8cc23bf2398a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.950794 28648 tablet_service.cc:1511] Processing CreateTablet for tablet e772ef1424fc4a8f8d9e3288e718e21a (DEFAULT_TABLE table=default.Table_1b1680a2f241430aa5d5702997655d36 [id=65c0597739b84caeb72d9d4c8a0636a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.950237 30890 raft_consensus.cc:493] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:45.951193 30890 raft_consensus.cc:515] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.951721 30890 leader_election.cc:290] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:45.952586 28518 tablet_service.cc:1511] Processing CreateTablet for tablet a1896e3b4a904fef8a741f63aa952251 (DEFAULT_TABLE table=default.Table_21242eba3cb64c7c91082e296439cfe2 [id=10a4f5ef579746b68d8dd37d83f14c3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.941903 28403 raft_consensus.cc:2468] T c200f34abf074684a0314c9024f04caf P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:45.952924 28518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1896e3b4a904fef8a741f63aa952251. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.954003 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dffeeb7313714c3f9eccc228c98ae7a3" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:45.955626 28618 leader_election.cc:343] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:45.956502 28644 tablet_service.cc:1511] Processing CreateTablet for tablet a1896e3b4a904fef8a741f63aa952251 (DEFAULT_TABLE table=default.Table_21242eba3cb64c7c91082e296439cfe2 [id=10a4f5ef579746b68d8dd37d83f14c3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.956652 28663 tablet_service.cc:1511] Processing CreateTablet for tablet aba68ae6805e4374b2688dd4f840b7c0 (DEFAULT_TABLE table=default.Table_fc6ead641f8e40de8744701bf5337457 [id=97016308846c44c7854c4962964d9b71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.957098 28519 tablet_service.cc:1511] Processing CreateTablet for tablet aa7faa3bb1b7474cb84e5f2a96af65e8 (DEFAULT_TABLE table=default.Table_62bf5ce52fb14bd089678663dd009a91 [id=713af0f85a434ed0bfb791c48f75dc9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.957424 28519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa7faa3bb1b7474cb84e5f2a96af65e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.957412 28512 tablet_service.cc:1511] Processing CreateTablet for tablet aba68ae6805e4374b2688dd4f840b7c0 (DEFAULT_TABLE table=default.Table_fc6ead641f8e40de8744701bf5337457 [id=97016308846c44c7854c4962964d9b71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.957749 28512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aba68ae6805e4374b2688dd4f840b7c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.957697 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dffeeb7313714c3f9eccc228c98ae7a3" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:45.958030 28403 raft_consensus.cc:2468] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:45.959569 28618 consensus_peers.cc:597] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:45.959790 28617 leader_election.cc:304] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.960424 30856 raft_consensus.cc:2804] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:45.960541 30856 raft_consensus.cc:697] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:45.960770 30856 consensus_queue.cc:237] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:45.961469 28653 tablet_service.cc:1511] Processing CreateTablet for tablet f683917aac5945ec939664ab82c24530 (DEFAULT_TABLE table=default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.961956 28658 tablet_service.cc:1511] Processing CreateTablet for tablet aa7faa3bb1b7474cb84e5f2a96af65e8 (DEFAULT_TABLE table=default.Table_62bf5ce52fb14bd089678663dd009a91 [id=713af0f85a434ed0bfb791c48f75dc9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.962980 28548 raft_consensus.cc:3060] T 8bef52c0fc354d6cb5388700641af9f8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.964447 28384 tablet_service.cc:1511] Processing CreateTablet for tablet aa7faa3bb1b7474cb84e5f2a96af65e8 (DEFAULT_TABLE table=default.Table_62bf5ce52fb14bd089678663dd009a91 [id=713af0f85a434ed0bfb791c48f75dc9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.964927 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa7faa3bb1b7474cb84e5f2a96af65e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.966682 28374 tablet_service.cc:1511] Processing CreateTablet for tablet f683917aac5945ec939664ab82c24530 (DEFAULT_TABLE table=default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.967063 28374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f683917aac5945ec939664ab82c24530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.968343 28527 tablet_service.cc:1511] Processing CreateTablet for tablet 105f61bdf2d541cab9423b6df285265e (DEFAULT_TABLE table=default.Table_73e7e7a1c4ad48d59b10333525a7d129 [id=a8ebdf3ba6824c5491ad24354b044009]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.969125 28660 tablet_service.cc:1511] Processing CreateTablet for tablet 931911d377ea4e3bbecd9346e081776a (DEFAULT_TABLE table=default.Table_e069e4a145044a9d908540306560f100 [id=008ff0d4acc5478aa5392ed66259e776]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.971161 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 7eb0471903fd4654af395f7e4b68eb99 (DEFAULT_TABLE table=default.Table_a90110674b24430c976945f2749a2c40 [id=b3dfe9538ef64f0e96814b63c0d90e5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.970806 28646 tablet_service.cc:1511] Processing CreateTablet for tablet 105f61bdf2d541cab9423b6df285265e (DEFAULT_TABLE table=default.Table_73e7e7a1c4ad48d59b10333525a7d129 [id=a8ebdf3ba6824c5491ad24354b044009]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.974145 28659 tablet_service.cc:1511] Processing CreateTablet for tablet 7eb0471903fd4654af395f7e4b68eb99 (DEFAULT_TABLE table=default.Table_a90110674b24430c976945f2749a2c40 [id=b3dfe9538ef64f0e96814b63c0d90e5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.979462 28390 tablet_service.cc:1511] Processing CreateTablet for tablet 931911d377ea4e3bbecd9346e081776a (DEFAULT_TABLE table=default.Table_e069e4a145044a9d908540306560f100 [id=008ff0d4acc5478aa5392ed66259e776]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.979717 28386 tablet_service.cc:1511] Processing CreateTablet for tablet 105f61bdf2d541cab9423b6df285265e (DEFAULT_TABLE table=default.Table_73e7e7a1c4ad48d59b10333525a7d129 [id=a8ebdf3ba6824c5491ad24354b044009]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.979918 28390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 931911d377ea4e3bbecd9346e081776a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.980086 28386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 105f61bdf2d541cab9423b6df285265e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.981448 28385 tablet_service.cc:1511] Processing CreateTablet for tablet 7eb0471903fd4654af395f7e4b68eb99 (DEFAULT_TABLE table=default.Table_a90110674b24430c976945f2749a2c40 [id=b3dfe9538ef64f0e96814b63c0d90e5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:45.981766 28385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eb0471903fd4654af395f7e4b68eb99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.984524 28617 leader_election.cc:304] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:45.985641 30572 raft_consensus.cc:2804] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:45.985719 30572 raft_consensus.cc:493] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:45.985777 30572 raft_consensus.cc:3060] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:45.992043 28458 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:45.999464 28527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 105f61bdf2d541cab9423b6df285265e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.999868 28511 tablet_service.cc:1511] Processing CreateTablet for tablet 931911d377ea4e3bbecd9346e081776a (DEFAULT_TABLE table=default.Table_e069e4a145044a9d908540306560f100 [id=008ff0d4acc5478aa5392ed66259e776]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:46.002542 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eb0471903fd4654af395f7e4b68eb99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.999449 28525 tablet_service.cc:1511] Processing CreateTablet for tablet f683917aac5945ec939664ab82c24530 (DEFAULT_TABLE table=default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:46.006847 28525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f683917aac5945ec939664ab82c24530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:45.996856 29017 catalog_manager.cc:5696] T 038ea0e9c0b04db8b7ec56b065e3cc51 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:46.010895 30622 raft_consensus.cc:493] T 8ac84641d0f1422193e42a42468789e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.010957 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:46.014544 29017 catalog_manager.cc:5696] T 7c50ab60a26647558da509236aac1e8b P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:46.013679 30572 raft_consensus.cc:515] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.018565 31048 tablet_bootstrap.cc:492] T 7c96a195e54440d4854e4462d178b468 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:46.019629 28546 raft_consensus.cc:3060] T ff8e487fd51f4aa18bf59e5f8e011dcd P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.019981 28375 tablet_service.cc:1511] Processing CreateTablet for tablet a1896e3b4a904fef8a741f63aa952251 (DEFAULT_TABLE table=default.Table_21242eba3cb64c7c91082e296439cfe2 [id=10a4f5ef579746b68d8dd37d83f14c3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:46.022851 28375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1896e3b4a904fef8a741f63aa952251. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:46.019135 30622 raft_consensus.cc:515] T 8ac84641d0f1422193e42a42468789e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.022235 31048 ts_tablet_manager.cc:1403] T 7c96a195e54440d4854e4462d178b468 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.107s user 0.000s sys 0.005s
I20260618 13:59:46.027948 30622 leader_election.cc:290] T 8ac84641d0f1422193e42a42468789e1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:46.028981 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ac84641d0f1422193e42a42468789e1" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:46.026707 30572 leader_election.cc:290] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.026180 28377 tablet_service.cc:1511] Processing CreateTablet for tablet aba68ae6805e4374b2688dd4f840b7c0 (DEFAULT_TABLE table=default.Table_fc6ead641f8e40de8744701bf5337457 [id=97016308846c44c7854c4962964d9b71]), partition=RANGE (key) PARTITION UNBOUNDED
W20260618 13:59:46.029817 28346 leader_election.cc:343] T 8ac84641d0f1422193e42a42468789e1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.030252 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dffeeb7313714c3f9eccc228c98ae7a3" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:46.030603 28377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aba68ae6805e4374b2688dd4f840b7c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:46.033057 28511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 931911d377ea4e3bbecd9346e081776a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:46.033202 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:46.035740 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55df96d305bf466b9fdea88a1816b2fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:46.036444 28659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eb0471903fd4654af395f7e4b68eb99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:46.032428 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ac84641d0f1422193e42a42468789e1" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.036724 28644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1896e3b4a904fef8a741f63aa952251. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:46.036940 28660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 931911d377ea4e3bbecd9346e081776a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:46.037094 28645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b51ac996e88453eb204e8b83f33863a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:46.037525 28646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 105f61bdf2d541cab9423b6df285265e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:46.042280 28648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e772ef1424fc4a8f8d9e3288e718e21a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:46.042519 28663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aba68ae6805e4374b2688dd4f840b7c0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:46.042949 28346 leader_election.cc:343] T 8ac84641d0f1422193e42a42468789e1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.043115 28346 leader_election.cc:304] T 8ac84641d0f1422193e42a42468789e1 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:46.043682 31056 raft_consensus.cc:2749] T 8ac84641d0f1422193e42a42468789e1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:46.030640 28408 raft_consensus.cc:3060] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.045946 28658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa7faa3bb1b7474cb84e5f2a96af65e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:46.047969 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]). Expected version 1 got 0
I20260618 13:59:46.048642 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:46.049036 29017 catalog_manager.cc:5696] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:46.049512 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet e196dd3c7fe14524a50152643ddbc2ad (table default.Table_2dfebd62a146446f999f1a7a1bd1af79 [id=02ffed3d852845d6845fb5176f036fce]). Expected version 1 got 0
I20260618 13:59:46.050129 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3a783590159f4a80964fbbcb4a24eef9 (table default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f]). Expected version 2 got 0
I20260618 13:59:46.050575 28653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f683917aac5945ec939664ab82c24530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:46.050855 29017 catalog_manager.cc:5696] T 51b91f053bcc42bb829bb8180e232c8e P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:46.051381 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:46.052245 30634 master_runner.cc:387] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.100.20:36797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.27.100.62:43001,127.27.100.61:45967,127.27.100.60:42083
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.100.60:42083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.100.60
--webserver_port=38293
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision faec34f53d4c9dd1e3a6dd2b7314d2fc7aadff6c
build type FASTDEBUG
built by None at 18 Jun 2026 13:43:31 UTC on bdcb31816ec0
build id 12454
ASAN enabled
I20260618 13:59:46.054349 30634 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 13:59:46.057719 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 00b602a473a94c05a617fb9478300c74 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:46.057927 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:46.058117 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:46.058524 30950 ts_tablet_manager.cc:1929] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:46.058542 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:46.058769 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:46.059163 29017 catalog_manager.cc:5696] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:46.059859 29017 catalog_manager.cc:5696] T b2a6fabad00642528913e735929e295e P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:46.060621 29017 catalog_manager.cc:5696] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:46.063383 29017 catalog_manager.cc:5696] T c5836982752b4209b72a372c1dfeb71c P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:46.064195 30634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20260618 13:59:46.065307 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 3f5d4296952e48a48756e6f27140da87 (table default.Table_01e1e5741bb54f78996857f6b79fc5d7 [id=3d09bbb90ecb4103b2460f7aafa2db48]). Expected version 2 got 0
I20260618 13:59:46.067732 29017 catalog_manager.cc:5696] T 5115a31a38db48c1a6474917aff97c8e P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:46.068225 30854 consensus_queue.cc:1048] T b44c3b0c152442a78edc4736da2cb15c P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:46.068559 30898 raft_consensus.cc:493] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.068737 30898 raft_consensus.cc:515] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.069635 30898 leader_election.cc:290] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.070887 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:46.072037 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dffeeb7313714c3f9eccc228c98ae7a3" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:46.072567 30950 log.cc:1199] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/91d014b46a0f448b929cbc9287ead0a7
I20260618 13:59:46.073698 28797 tablet_bootstrap.cc:492] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:46.073908 28797 ts_tablet_manager.cc:1403] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.201s user 0.006s sys 0.010s
I20260618 13:59:46.066473 31048 raft_consensus.cc:359] T 7c96a195e54440d4854e4462d178b468 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.074795 31048 raft_consensus.cc:385] T 7c96a195e54440d4854e4462d178b468 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.074877 31048 raft_consensus.cc:740] T 7c96a195e54440d4854e4462d178b468 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.075120 31048 consensus_queue.cc:260] T 7c96a195e54440d4854e4462d178b468 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.075663 31048 ts_tablet_manager.cc:1434] T 7c96a195e54440d4854e4462d178b468 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.048s user 0.000s sys 0.001s
I20260618 13:59:46.075954 31048 tablet_bootstrap.cc:492] T 5b51ac996e88453eb204e8b83f33863a P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:46.076292 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:46.076990 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a9a5f630654465fb416f53af1424e9b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:46.077169 28404 raft_consensus.cc:2468] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:46.077600 28617 leader_election.cc:304] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.077095 29017 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:46.078612 30898 raft_consensus.cc:2804] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:46.078735 30898 raft_consensus.cc:493] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:46.078837 30898 raft_consensus.cc:3060] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.081399 30898 raft_consensus.cc:515] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.081820 28797 raft_consensus.cc:359] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.081970 28797 raft_consensus.cc:385] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.082034 28797 raft_consensus.cc:740] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.082202 28797 consensus_queue.cc:260] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.082717 30898 leader_election.cc:290] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.084489 29026 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet ed648dcf882340388155068d7a5a7869
I20260618 13:59:46.084982 29026 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 137) took 4595 ms. Trace:
I20260618 13:59:46.085325 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a9a5f630654465fb416f53af1424e9b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:46.085479 28404 raft_consensus.cc:3060] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.085063 29026 rpcz_store.cc:276] 0618 13:59:41.489249 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:44.961197 (+3471948us) service_pool.cc:224] Handling call
0618 13:59:44.961891 (+ 694us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:44.962079 (+ 188us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:44.962160 (+ 81us) catalog_manager.cc:2849] Locking tablets
0618 13:59:44.962338 (+ 178us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:45.011055 (+ 48717us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:45.046756 (+ 35701us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:46.084441 (+1037685us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:46.084930 (+ 489us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:44.995171 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:44.995291 (+ 120us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:44.995298 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:44.995300 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:44.995420 (+ 120us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:44.995436 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:44.995441 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:44.995563 (+ 122us) tablet.cc:701] Row locks acquired
0618 13:59:44.995576 (+ 13us) write_op.cc:260] PREPARE: finished
0618 13:59:44.995611 (+ 35us) write_op.cc:270] Start()
0618 13:59:44.995644 (+ 33us) write_op.cc:276] Timestamp: P: 1781791184995590 usec, L: 0
0618 13:59:44.995647 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:44.995822 (+ 175us) log.cc:844] Serialized 762 byte log entry
0618 13:59:45.008617 (+ 12795us) op_driver.cc:464] REPLICATION: finished
0618 13:59:45.008767 (+ 150us) write_op.cc:301] APPLY: starting
0618 13:59:45.008785 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:45.008835 (+ 50us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:45.008874 (+ 39us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:45.008947 (+ 73us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:45.008982 (+ 35us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:45.008989 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:45.009074 (+ 85us) log.cc:844] Serialized 29 byte log entry
0618 13:59:45.009141 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.009164 (+ 23us) write_op.cc:454] Released schema lock
0618 13:59:45.009172 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:45.009244 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.009248 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":98,"apply.run_cpu_time_us":468,"apply.run_wall_time_us":467,"num_ops":2,"prepare.queue_time_us":54,"prepare.run_cpu_time_us":756,"prepare.run_wall_time_us":756,"raft.queue_time_us":145,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":260,"replication_time_us":12935}]]}
I20260618 13:59:46.085901 29026 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_8f6f0bc5df454e3e9e307c07a4b86c53" } modify_external_catalogs: true
I20260618 13:59:46.086068 29026 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_8f6f0bc5df454e3e9e307c07a4b86c53" } modify_external_catalogs: true
I20260618 13:59:46.048521 28408 raft_consensus.cc:2468] T dffeeb7313714c3f9eccc228c98ae7a3 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:46.087965 28797 ts_tablet_manager.cc:1434] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20260618 13:59:46.088248 28797 tablet_bootstrap.cc:492] T 2128dbec33db453ba51070767a15b21e P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:46.088459 28404 raft_consensus.cc:2468] T 4a9a5f630654465fb416f53af1424e9b P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:46.089428 31057 consensus_queue.cc:1048] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:46.089851 28797 tablet_bootstrap.cc:654] T 2128dbec33db453ba51070767a15b21e P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.091038 28659 tablet_service.cc:1558] Processing DeleteTablet for tablet ed648dcf882340388155068d7a5a7869 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:44 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:46.092077 29566 tablet_replica.cc:333] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:46.092336 29566 raft_consensus.cc:2243] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:46.094441 28617 leader_election.cc:304] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.094743 28617 leader_election.cc:304] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.096081 30572 raft_consensus.cc:2804] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:46.096962 30890 raft_consensus.cc:2804] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:46.097075 30890 raft_consensus.cc:697] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:46.097295 30890 consensus_queue.cc:237] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.097380 29566 raft_consensus.cc:2272] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:46.098050 29566 ts_tablet_manager.cc:1916] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:46.100509 30572 raft_consensus.cc:697] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:46.100881 30572 consensus_queue.cc:237] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.102713 31048 tablet_bootstrap.cc:654] T 5b51ac996e88453eb204e8b83f33863a P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.104458 29019 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 39b06ae7444a48dbbec9e096fd7e4052
I20260618 13:59:46.105111 29019 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 155) took 2851 ms. Trace:
I20260618 13:59:46.105228 29019 rpcz_store.cc:276] 0618 13:59:43.254023 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:45.276175 (+2022152us) service_pool.cc:224] Handling call
0618 13:59:45.286092 (+ 9917us) spinlock_profiling.cc:243] Waited 9.46 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb46856b6 00007fcbab0e2486 00007fcbab0e21b5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:45.286633 (+ 541us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:45.286764 (+ 131us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:45.286833 (+ 69us) catalog_manager.cc:2849] Locking tablets
0618 13:59:45.462265 (+175432us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:45.468162 (+ 5897us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:45.468243 (+ 81us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:46.104360 (+636117us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:46.105075 (+ 715us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:45.463213 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:45.463361 (+ 148us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:45.463369 (+ 8us) write_op.cc:435] Acquired schema lock
0618 13:59:45.463373 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:45.463514 (+ 141us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:45.463529 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:45.463536 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:45.463658 (+ 122us) tablet.cc:701] Row locks acquired
0618 13:59:45.463662 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:45.463698 (+ 36us) write_op.cc:270] Start()
0618 13:59:45.463737 (+ 39us) write_op.cc:276] Timestamp: P: 1781791185463627 usec, L: 0
0618 13:59:45.463742 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:45.463957 (+ 215us) log.cc:844] Serialized 768 byte log entry
0618 13:59:45.467284 (+ 3327us) op_driver.cc:464] REPLICATION: finished
0618 13:59:45.467443 (+ 159us) write_op.cc:301] APPLY: starting
0618 13:59:45.467459 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:45.467512 (+ 53us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:45.467536 (+ 24us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:45.467597 (+ 61us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:45.467627 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:45.467635 (+ 8us) write_op.cc:312] APPLY: finished
0618 13:59:45.467700 (+ 65us) log.cc:844] Serialized 29 byte log entry
0618 13:59:45.467752 (+ 52us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.467771 (+ 19us) write_op.cc:454] Released schema lock
0618 13:59:45.467776 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:45.467881 (+ 105us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.467885 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":21752704,"child_traces":[["op",{"apply.queue_time_us":118,"apply.run_cpu_time_us":426,"apply.run_wall_time_us":426,"num_ops":2,"prepare.queue_time_us":58,"prepare.run_cpu_time_us":862,"prepare.run_wall_time_us":863,"raft.queue_time_us":224,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":205,"replication_time_us":3516}]]}
I20260618 13:59:46.106096 29019 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_bc807592fe5a4a1d8ce8cbe03c0e4f5a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:46.106520 31048 tablet_bootstrap.cc:492] T 5b51ac996e88453eb204e8b83f33863a P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:46.106647 31048 ts_tablet_manager.cc:1403] T 5b51ac996e88453eb204e8b83f33863a P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.001s
W20260618 13:59:46.106721 29019 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc807592fe5a4a1d8ce8cbe03c0e4f5a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:46.107470 31048 raft_consensus.cc:359] T 5b51ac996e88453eb204e8b83f33863a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.107625 31048 raft_consensus.cc:385] T 5b51ac996e88453eb204e8b83f33863a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.107702 31048 raft_consensus.cc:740] T 5b51ac996e88453eb204e8b83f33863a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.107995 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet ed648dcf882340388155068d7a5a7869 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:44 UTC) from {username='slave'} at 127.0.0.1:57282
W20260618 13:59:46.111025 28988 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet 72c04f55890648f0a8184f3783243f7c (table default.Table_773be314d7e741fcb4fb6eb7d954802e [id=0d83fc68255c458587390c58b12e00d7], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 72c04f55890648f0a8184f3783243f7c consensus state has no leader
I20260618 13:59:46.116529 28797 tablet_bootstrap.cc:492] T 2128dbec33db453ba51070767a15b21e P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:46.116671 28797 ts_tablet_manager.cc:1403] T 2128dbec33db453ba51070767a15b21e P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.001s
I20260618 13:59:46.117419 28797 raft_consensus.cc:359] T 2128dbec33db453ba51070767a15b21e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.117597 28797 raft_consensus.cc:385] T 2128dbec33db453ba51070767a15b21e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.117652 28797 raft_consensus.cc:740] T 2128dbec33db453ba51070767a15b21e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.117848 28797 consensus_queue.cc:260] T 2128dbec33db453ba51070767a15b21e P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.118826 31048 consensus_queue.cc:260] T 5b51ac996e88453eb204e8b83f33863a P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.119340 31048 ts_tablet_manager.cc:1434] T 5b51ac996e88453eb204e8b83f33863a P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.013s user 0.001s sys 0.000s
I20260618 13:59:46.120518 29019 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 157) took 2825 ms. Trace:
I20260618 13:59:46.120584 28377 tablet_service.cc:1558] Processing DeleteTablet for tablet ed648dcf882340388155068d7a5a7869 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:44 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:46.120709 29019 rpcz_store.cc:276] 0618 13:59:43.295223 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:46.105890 (+2810667us) service_pool.cc:224] Handling call
0618 13:59:46.106805 (+ 915us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:46.107305 (+ 500us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:46.117348 (+ 10043us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:46.117681 (+ 333us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:46.120472 (+ 2791us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:46.109933 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:46.110058 (+ 125us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:46.110065 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:46.110068 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:46.110163 (+ 95us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:46.110177 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:46.110181 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:46.110255 (+ 74us) tablet.cc:701] Row locks acquired
0618 13:59:46.110258 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:46.110293 (+ 35us) write_op.cc:270] Start()
0618 13:59:46.110325 (+ 32us) write_op.cc:276] Timestamp: P: 1781791186110270 usec, L: 0
0618 13:59:46.110329 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:46.110542 (+ 213us) log.cc:844] Serialized 433 byte log entry
0618 13:59:46.114262 (+ 3720us) op_driver.cc:464] REPLICATION: finished
0618 13:59:46.114393 (+ 131us) write_op.cc:301] APPLY: starting
0618 13:59:46.114408 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:46.114483 (+ 75us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:46.114530 (+ 47us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:46.114664 (+ 134us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:46.114697 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:46.114703 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:46.114776 (+ 73us) log.cc:844] Serialized 29 byte log entry
0618 13:59:46.114839 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:46.114865 (+ 26us) write_op.cc:454] Released schema lock
0618 13:59:46.114874 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:46.114964 (+ 90us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:46.114971 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":89,"apply.run_cpu_time_us":549,"apply.run_wall_time_us":548,"num_ops":2,"prepare.queue_time_us":2081,"prepare.run_cpu_time_us":715,"prepare.run_wall_time_us":714,"raft.queue_time_us":161,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":197,"replication_time_us":3906}]]}
I20260618 13:59:46.121912 29019 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_0b63e77c05484ae28bd843613ed1b0cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:46.122133 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet 39b06ae7444a48dbbec9e096fd7e4052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:45 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:46.123811 31010 tablet.cc:1724] T d15b1b603f024f3295eb07c98991076d P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:46.125121 28651 tablet_service.cc:1558] Processing DeleteTablet for tablet 39b06ae7444a48dbbec9e096fd7e4052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:45 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:46.131063 31007 tablet.cc:1724] T d15b1b603f024f3295eb07c98991076d P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:46.131480 28797 ts_tablet_manager.cc:1434] T 2128dbec33db453ba51070767a15b21e P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.015s user 0.001s sys 0.001s
I20260618 13:59:46.131769 28797 tablet_bootstrap.cc:492] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:46.133464 28797 tablet_bootstrap.cc:654] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:46.136238 29019 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0b63e77c05484ae28bd843613ed1b0cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:46.136951 28346 consensus_peers.cc:597] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:46.143406 29566 ts_tablet_manager.cc:1929] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:46.143533 29566 log.cc:1199] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/ed648dcf882340388155068d7a5a7869
I20260618 13:59:46.143932 29566 ts_tablet_manager.cc:1950] T ed648dcf882340388155068d7a5a7869 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:46.144373 29019 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 158) took 2764 ms. Trace:
I20260618 13:59:46.145159 29019 rpcz_store.cc:276] 0618 13:59:43.379960 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:46.121656 (+2741696us) service_pool.cc:224] Handling call
0618 13:59:46.136380 (+ 14724us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:46.136868 (+ 488us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:46.144087 (+ 7219us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:46.144092 (+ 5us) write_op.cc:454] Released schema lock
0618 13:59:46.144155 (+ 63us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:46.144252 (+ 97us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:46.144335 (+ 83us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:46.137406 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:46.137548 (+ 142us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:46.137557 (+ 9us) write_op.cc:435] Acquired schema lock
0618 13:59:46.137561 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:46.137691 (+ 130us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:46.137709 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:46.137716 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:46.137815 (+ 99us) tablet.cc:701] Row locks acquired
0618 13:59:46.137819 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:46.137858 (+ 39us) write_op.cc:270] Start()
0618 13:59:46.137898 (+ 40us) write_op.cc:276] Timestamp: P: 1781791186137829 usec, L: 0
0618 13:59:46.137903 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:46.138077 (+ 174us) log.cc:844] Serialized 433 byte log entry
0618 13:59:46.141181 (+ 3104us) op_driver.cc:464] REPLICATION: finished
0618 13:59:46.141315 (+ 134us) write_op.cc:301] APPLY: starting
0618 13:59:46.141330 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:46.141373 (+ 43us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:46.141398 (+ 25us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:46.141508 (+ 110us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:46.141541 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:46.141548 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:46.141629 (+ 81us) log.cc:844] Serialized 29 byte log entry
0618 13:59:46.141685 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:46.141704 (+ 19us) write_op.cc:454] Released schema lock
0618 13:59:46.141710 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":95,"apply.run_cpu_time_us":440,"apply.run_wall_time_us":440,"num_ops":2,"prepare.queue_time_us":51,"prepare.run_cpu_time_us":770,"prepare.run_wall_time_us":769,"raft.queue_time_us":146,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":319,"replication_time_us":3251}]]}
I20260618 13:59:46.146289 30847 consensus_queue.cc:1048] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:46.146927 29019 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_330e421310fd41adba536e77619ab0ad" } new_table_name: "default.Table_d99f88d5cf4743e1859bc0a151a83b90" modify_external_catalogs: true
I20260618 13:59:46.147387 29566 tablet_replica.cc:333] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:46.147521 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet ed648dcf882340388155068d7a5a7869 (table default.Table_8c9c9a6eea174bd68d8fb9d9ebaaf603 [id=185e7792e79644a2aced2b949fca635b]) successfully deleted
I20260618 13:59:46.147595 29566 raft_consensus.cc:2243] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:46.150666 29011 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 151) took 3303 ms. Trace:
I20260618 13:59:46.150776 29011 rpcz_store.cc:276] 0618 13:59:42.846784 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:45.200243 (+2353459us) service_pool.cc:224] Handling call
0618 13:59:45.200687 (+ 444us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:45.200914 (+ 227us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:45.201355 (+ 441us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:45.201446 (+ 91us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:45.243296 (+ 41850us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:46.150604 (+907308us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:45.201999 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:45.202118 (+ 119us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:45.202125 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:45.202127 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:45.202225 (+ 98us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:45.202239 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:45.202244 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:45.202313 (+ 69us) tablet.cc:701] Row locks acquired
0618 13:59:45.202315 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:45.202354 (+ 39us) write_op.cc:270] Start()
0618 13:59:45.202386 (+ 32us) write_op.cc:276] Timestamp: P: 1781791185202310 usec, L: 0
0618 13:59:45.202389 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:45.202635 (+ 246us) log.cc:844] Serialized 405 byte log entry
0618 13:59:45.238998 (+ 36363us) op_driver.cc:464] REPLICATION: finished
0618 13:59:45.239706 (+ 708us) write_op.cc:301] APPLY: starting
0618 13:59:45.239729 (+ 23us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:45.239777 (+ 48us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:45.239811 (+ 34us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:45.239874 (+ 63us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:45.239910 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:45.239917 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:45.239997 (+ 80us) log.cc:844] Serialized 23 byte log entry
0618 13:59:45.240076 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.240095 (+ 19us) write_op.cc:454] Released schema lock
0618 13:59:45.240103 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:45.240184 (+ 81us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.240189 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":643,"apply.run_cpu_time_us":472,"apply.run_wall_time_us":471,"num_ops":1,"prepare.queue_time_us":40,"prepare.run_cpu_time_us":731,"prepare.run_wall_time_us":731,"raft.queue_time_us":560,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":225,"replication_time_us":36576}]]}
I20260618 13:59:46.151306 29566 raft_consensus.cc:2272] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:46.151652 30854 raft_consensus.cc:493] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.151794 29566 ts_tablet_manager.cc:1916] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:46.151757 30854 raft_consensus.cc:515] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.152390 30939 tablet_replica.cc:333] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:46.152437 31048 tablet_bootstrap.cc:492] T e772ef1424fc4a8f8d9e3288e718e21a P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:46.152652 30939 raft_consensus.cc:2243] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:46.152942 30939 raft_consensus.cc:2272] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:46.153517 30939 ts_tablet_manager.cc:1916] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:46.154114 31048 tablet_bootstrap.cc:654] T e772ef1424fc4a8f8d9e3288e718e21a P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.161880 30939 ts_tablet_manager.cc:1929] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:46.163457 29566 ts_tablet_manager.cc:1929] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:46.164754 31056 raft_consensus.cc:493] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.167986 30939 log.cc:1199] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/ed648dcf882340388155068d7a5a7869
I20260618 13:59:46.168001 31056 raft_consensus.cc:515] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.168363 30939 ts_tablet_manager.cc:1950] T ed648dcf882340388155068d7a5a7869 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:46.168761 31056 leader_election.cc:290] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:46.169184 30939 tablet_replica.cc:333] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:46.169818 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet ed648dcf882340388155068d7a5a7869 (table default.Table_8c9c9a6eea174bd68d8fb9d9ebaaf603 [id=185e7792e79644a2aced2b949fca635b]) successfully deleted
I20260618 13:59:46.170604 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60e02f51b8624a739fadbf4fafe90cb5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.169390 30939 raft_consensus.cc:2243] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:46.173928 30939 raft_consensus.cc:2272] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:46.173990 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43224c621a4774870d44f0237b514a" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:46.174140 28404 raft_consensus.cc:2468] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:46.174504 30939 ts_tablet_manager.cc:1916] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:46.176055 28346 leader_election.cc:343] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.178407 30950 ts_tablet_manager.cc:1950] T 91d014b46a0f448b929cbc9287ead0a7 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:46.185225 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 91d014b46a0f448b929cbc9287ead0a7 (table default.Table_62bf5ce52fb14bd089678663dd009a91 [id=713af0f85a434ed0bfb791c48f75dc9d]) successfully deleted
W20260618 13:59:46.185658 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet c9ceb3af08fa4618b81cf6f89d7bd0df because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c9ceb3af08fa4618b81cf6f89d7bd0df already in progress: creating tablet
W20260618 13:59:46.185993 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet 60b6216474b74a14af6abf977e312b77 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60b6216474b74a14af6abf977e312b77 already in progress: creating tablet
I20260618 13:59:46.192047 30939 ts_tablet_manager.cc:1929] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:46.192270 30939 log.cc:1199] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/39b06ae7444a48dbbec9e096fd7e4052
I20260618 13:59:46.192796 30939 ts_tablet_manager.cc:1950] T 39b06ae7444a48dbbec9e096fd7e4052 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:46.194272 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 39b06ae7444a48dbbec9e096fd7e4052 (table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15]) successfully deleted
I20260618 13:59:46.198350 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 39b06ae7444a48dbbec9e096fd7e4052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:45 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:46.201143 29566 log.cc:1199] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/39b06ae7444a48dbbec9e096fd7e4052
I20260618 13:59:46.165061 28797 tablet_bootstrap.cc:492] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:46.166465 30854 leader_election.cc:290] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.169840 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60e02f51b8624a739fadbf4fafe90cb5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:46.181572 28617 leader_election.cc:304] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.199419 30847 raft_consensus.cc:493] T 2128dbec33db453ba51070767a15b21e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.201472 28797 ts_tablet_manager.cc:1403] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.070s user 0.003s sys 0.000s
I20260618 13:59:46.201529 28682 raft_consensus.cc:2468] T 60e02f51b8624a739fadbf4fafe90cb5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:46.201531 30847 raft_consensus.cc:515] T 2128dbec33db453ba51070767a15b21e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.202148 30847 leader_election.cc:290] T 2128dbec33db453ba51070767a15b21e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.202267 28797 raft_consensus.cc:359] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.202490 28797 raft_consensus.cc:385] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.202540 28797 raft_consensus.cc:740] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.202765 28797 consensus_queue.cc:260] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.203292 28797 ts_tablet_manager.cc:1434] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:46.203565 28797 tablet_bootstrap.cc:492] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:46.203509 28346 leader_election.cc:304] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:46.203816 31056 raft_consensus.cc:2804] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:46.203892 31056 raft_consensus.cc:493] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:46.203962 31056 raft_consensus.cc:3060] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.204095 30847 raft_consensus.cc:2804] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:46.204165 30847 raft_consensus.cc:493] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:46.204236 30847 raft_consensus.cc:3060] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.205385 28797 tablet_bootstrap.cc:654] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.206596 31056 raft_consensus.cc:515] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.207166 31056 leader_election.cc:290] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:46.207715 30847 raft_consensus.cc:515] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.208359 30847 leader_election.cc:290] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.208982 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60e02f51b8624a739fadbf4fafe90cb5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:46.209355 28651 tablet_service.cc:1511] Processing CreateTablet for tablet 9a92ea63f4f842419d6b0a5d638cca88 (DEFAULT_TABLE table=default.Table_bc807592fe5a4a1d8ce8cbe03c0e4f5a [id=505eda4290d644f8b9d8e73da1730089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:46.209775 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a92ea63f4f842419d6b0a5d638cca88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:46.215040 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43224c621a4774870d44f0237b514a" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:46.215216 28404 raft_consensus.cc:3060] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.216148 28682 raft_consensus.cc:3060] T 60e02f51b8624a739fadbf4fafe90cb5 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.217842 28404 raft_consensus.cc:2468] T 7d43224c621a4774870d44f0237b514a P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:46.218686 28682 raft_consensus.cc:2468] T 60e02f51b8624a739fadbf4fafe90cb5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:46.219203 28346 leader_election.cc:304] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.219472 31056 raft_consensus.cc:2804] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:46.219635 31056 raft_consensus.cc:697] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:46.219892 31056 consensus_queue.cc:237] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:46.222108 28617 leader_election.cc:343] T 2128dbec33db453ba51070767a15b21e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 2128dbec33db453ba51070767a15b21e
I20260618 13:59:46.222286 28617 leader_election.cc:304] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.226658 30847 raft_consensus.cc:2804] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:46.227257 30847 raft_consensus.cc:697] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:46.227555 30847 consensus_queue.cc:237] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.227782 28517 tablet_service.cc:1511] Processing CreateTablet for tablet 9a92ea63f4f842419d6b0a5d638cca88 (DEFAULT_TABLE table=default.Table_bc807592fe5a4a1d8ce8cbe03c0e4f5a [id=505eda4290d644f8b9d8e73da1730089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:46.228260 28517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a92ea63f4f842419d6b0a5d638cca88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:46.229127 29566 ts_tablet_manager.cc:1950] T 39b06ae7444a48dbbec9e096fd7e4052 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:46.232388 31010 tablet.cc:1724] T 957670718c8c4b8cb87115cc4a2fa861 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:46.232395 28797 tablet_bootstrap.cc:492] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:46.233729 28797 ts_tablet_manager.cc:1403] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.006s
I20260618 13:59:46.234513 28797 raft_consensus.cc:359] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.234680 28797 raft_consensus.cc:385] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.234738 28797 raft_consensus.cc:740] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.234925 28797 consensus_queue.cc:260] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.235458 28797 ts_tablet_manager.cc:1434] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:46.235700 28797 tablet_bootstrap.cc:492] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:46.239084 31007 tablet.cc:1724] T 957670718c8c4b8cb87115cc4a2fa861 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:46.246466 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 39b06ae7444a48dbbec9e096fd7e4052 (table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15]) successfully deleted
I20260618 13:59:46.249195 28797 tablet_bootstrap.cc:654] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.256856 28375 tablet_service.cc:1511] Processing CreateTablet for tablet 9a92ea63f4f842419d6b0a5d638cca88 (DEFAULT_TABLE table=default.Table_bc807592fe5a4a1d8ce8cbe03c0e4f5a [id=505eda4290d644f8b9d8e73da1730089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:46.257359 28375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a92ea63f4f842419d6b0a5d638cca88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:46.260149 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a9a5f630654465fb416f53af1424e9b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.260160 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a9a5f630654465fb416f53af1424e9b" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:46.264868 28545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60e02f51b8624a739fadbf4fafe90cb5" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:46.265342 28346 leader_election.cc:343] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.266301 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43224c621a4774870d44f0237b514a" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.267139 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2128dbec33db453ba51070767a15b21e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.273949 30847 consensus_queue.cc:1048] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:46.277712 28797 tablet_bootstrap.cc:492] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:46.279600 28797 ts_tablet_manager.cc:1403] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.044s user 0.002s sys 0.002s
I20260618 13:59:46.280406 28797 raft_consensus.cc:359] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.280567 28797 raft_consensus.cc:385] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.280618 28797 raft_consensus.cc:740] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.284570 28797 consensus_queue.cc:260] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.285131 28797 ts_tablet_manager.cc:1434] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260618 13:59:46.289491 28797 tablet_bootstrap.cc:492] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:46.289822 28798 tablet_bootstrap.cc:492] T 60b6216474b74a14af6abf977e312b77 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:46.289969 28798 ts_tablet_manager.cc:1403] T 60b6216474b74a14af6abf977e312b77 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.455s user 0.005s sys 0.009s
I20260618 13:59:46.290037 28549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43224c621a4774870d44f0237b514a" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:46.291742 30950 tablet_replica.cc:333] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:46.297005 28797 tablet_bootstrap.cc:654] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.308571 31048 tablet_bootstrap.cc:492] T e772ef1424fc4a8f8d9e3288e718e21a P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:46.308710 31048 ts_tablet_manager.cc:1403] T e772ef1424fc4a8f8d9e3288e718e21a P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.156s user 0.010s sys 0.021s
I20260618 13:59:46.309953 28545 raft_consensus.cc:3060] T d8a71e334ed1400d97c965b49b677a81 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.311172 28798 raft_consensus.cc:359] T 60b6216474b74a14af6abf977e312b77 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.311347 28798 raft_consensus.cc:385] T 60b6216474b74a14af6abf977e312b77 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.311399 28798 raft_consensus.cc:740] T 60b6216474b74a14af6abf977e312b77 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.311661 28798 consensus_queue.cc:260] T 60b6216474b74a14af6abf977e312b77 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.312506 28730 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:46.316526 28798 ts_tablet_manager.cc:1434] T 60b6216474b74a14af6abf977e312b77 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.026s user 0.001s sys 0.001s
I20260618 13:59:46.317090 28798 tablet_bootstrap.cc:492] T e196dd3c7fe14524a50152643ddbc2ad P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:46.321705 30950 raft_consensus.cc:2243] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:46.322024 30950 raft_consensus.cc:2272] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:46.323104 30950 ts_tablet_manager.cc:1916] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:46.330564 28618 leader_election.cc:343] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:46.331315 28618 leader_election.cc:343] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:46.331733 28618 leader_election.cc:343] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:46.333348 28618 leader_election.cc:343] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.309495 31048 raft_consensus.cc:359] T e772ef1424fc4a8f8d9e3288e718e21a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.335800 31048 raft_consensus.cc:385] T e772ef1424fc4a8f8d9e3288e718e21a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.335886 31048 raft_consensus.cc:740] T e772ef1424fc4a8f8d9e3288e718e21a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.336820 28797 tablet_bootstrap.cc:492] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:46.336998 28797 ts_tablet_manager.cc:1403] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.052s user 0.004s sys 0.000s
I20260618 13:59:46.337772 28797 raft_consensus.cc:359] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.337937 28797 raft_consensus.cc:385] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.337996 28797 raft_consensus.cc:740] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.338369 31048 consensus_queue.cc:260] T e772ef1424fc4a8f8d9e3288e718e21a P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.338766 31056 raft_consensus.cc:493] T 7c96a195e54440d4854e4462d178b468 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.338917 31056 raft_consensus.cc:515] T 7c96a195e54440d4854e4462d178b468 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.339504 31056 leader_election.cc:290] T 7c96a195e54440d4854e4462d178b468 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:46.339612 28797 consensus_queue.cc:260] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.339741 30856 raft_consensus.cc:493] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.339850 30856 raft_consensus.cc:515] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.340255 28797 ts_tablet_manager.cc:1434] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 13:59:46.340387 30856 leader_election.cc:290] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.340859 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc88bf849a94cb9bd1c9c14aec53eb1" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:46.341171 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c96a195e54440d4854e4462d178b468" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.341255 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c96a195e54440d4854e4462d178b468" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:46.341440 31048 ts_tablet_manager.cc:1434] T e772ef1424fc4a8f8d9e3288e718e21a P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.033s user 0.003s sys 0.000s
I20260618 13:59:46.341464 28411 raft_consensus.cc:2468] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
W20260618 13:59:46.341682 28346 leader_election.cc:343] T 7c96a195e54440d4854e4462d178b468 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.341955 30856 raft_consensus.cc:493] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.341980 28617 leader_election.cc:304] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.342065 30856 raft_consensus.cc:515] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.342216 30572 raft_consensus.cc:2804] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:46.342283 30572 raft_consensus.cc:493] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:46.342340 30572 raft_consensus.cc:3060] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.342613 30856 leader_election.cc:290] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.342617 28797 tablet_bootstrap.cc:492] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:46.342932 28346 leader_election.cc:343] T 7c96a195e54440d4854e4462d178b468 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.343048 28346 leader_election.cc:304] T 7c96a195e54440d4854e4462d178b468 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:46.343087 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daface32f2a94a49a2db0729dfba32c2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:46.343233 28403 raft_consensus.cc:2468] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:46.343572 31056 raft_consensus.cc:2749] T 7c96a195e54440d4854e4462d178b468 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:46.343601 28617 leader_election.cc:304] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.343909 30856 raft_consensus.cc:2804] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:46.343986 30856 raft_consensus.cc:493] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:46.344048 30856 raft_consensus.cc:3060] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.344660 31048 tablet_bootstrap.cc:492] T 105f61bdf2d541cab9423b6df285265e P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:46.346042 28540 raft_consensus.cc:3060] T a8da96fc621c47d4a444b5a39259f85d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.347215 30856 raft_consensus.cc:515] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.347729 30856 leader_election.cc:290] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:46.347890 28618 leader_election.cc:343] T 2128dbec33db453ba51070767a15b21e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.348029 28618 leader_election.cc:304] T 2128dbec33db453ba51070767a15b21e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
W20260618 13:59:46.348543 28618 leader_election.cc:343] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.349093 30572 raft_consensus.cc:515] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.349283 30856 raft_consensus.cc:2749] T 2128dbec33db453ba51070767a15b21e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:46.349639 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daface32f2a94a49a2db0729dfba32c2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
W20260618 13:59:46.349710 28618 consensus_peers.cc:597] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:46.349869 28411 raft_consensus.cc:3060] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:46.350086 28618 consensus_peers.cc:597] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:46.350261 28618 consensus_peers.cc:597] T b89b26d1595642ff82381f8baddfb99a P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:46.350975 31067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 13:59:46.351222 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc88bf849a94cb9bd1c9c14aec53eb1" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:46.351357 28403 raft_consensus.cc:3060] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.351449 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc88bf849a94cb9bd1c9c14aec53eb1" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.347254 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daface32f2a94a49a2db0729dfba32c2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.351670 30572 leader_election.cc:290] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.353015 30986 tablet.cc:1724] T d15b1b603f024f3295eb07c98991076d P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:46.364528 28797 tablet_bootstrap.cc:654] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.365489 28798 tablet_bootstrap.cc:654] T e196dd3c7fe14524a50152643ddbc2ad P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.368945 31087 raft_consensus.cc:493] T c25fa765ec794c34b0aa43761961a8c9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.369139 31087 raft_consensus.cc:515] T c25fa765ec794c34b0aa43761961a8c9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.370527 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c25fa765ec794c34b0aa43761961a8c9" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:46.373107 31087 leader_election.cc:290] T c25fa765ec794c34b0aa43761961a8c9 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:46.377327 31085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 13:59:46.379537 28403 raft_consensus.cc:2468] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:46.379644 28411 raft_consensus.cc:2468] T daface32f2a94a49a2db0729dfba32c2 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
W20260618 13:59:46.374356 28618 leader_election.cc:343] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.380012 28617 leader_election.cc:304] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.384572 28617 leader_election.cc:304] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.384871 30572 raft_consensus.cc:2804] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:46.385017 30572 raft_consensus.cc:697] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:46.385087 30856 raft_consensus.cc:2804] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:46.385259 30572 consensus_queue.cc:237] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.385533 30856 raft_consensus.cc:697] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:46.385792 30856 consensus_queue.cc:237] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.385915 31075 raft_consensus.cc:493] T d8a71e334ed1400d97c965b49b677a81 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6a169b16ced46ca9cfbc631709bb257)
I20260618 13:59:46.386034 31075 raft_consensus.cc:515] T d8a71e334ed1400d97c965b49b677a81 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.389365 31080 raft_consensus.cc:493] T 60b6216474b74a14af6abf977e312b77 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.389544 31080 raft_consensus.cc:515] T 60b6216474b74a14af6abf977e312b77 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:46.389878 28618 leader_election.cc:343] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.391556 30890 raft_consensus.cc:493] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.391729 30890 raft_consensus.cc:515] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.393335 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7c6c17ba06543219259749d7d38832d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:46.393522 28411 raft_consensus.cc:2468] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:46.394117 28538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "daface32f2a94a49a2db0729dfba32c2" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:46.394943 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbc88bf849a94cb9bd1c9c14aec53eb1" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:46.394956 30552 raft_consensus.cc:493] T ff8e487fd51f4aa18bf59e5f8e011dcd P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6fc808af0e884ed8836561665d8e0e7b)
I20260618 13:59:46.395143 30552 raft_consensus.cc:515] T ff8e487fd51f4aa18bf59e5f8e011dcd P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.395547 31080 leader_election.cc:290] T 60b6216474b74a14af6abf977e312b77 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.395808 30552 leader_election.cc:290] T ff8e487fd51f4aa18bf59e5f8e011dcd P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.396214 28617 leader_election.cc:304] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.396498 30890 leader_election.cc:290] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.396699 30890 raft_consensus.cc:2804] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:46.396682 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8e487fd51f4aa18bf59e5f8e011dcd" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:46.396821 30890 raft_consensus.cc:493] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:46.396929 30890 raft_consensus.cc:3060] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:46.398531 28483 leader_election.cc:343] T 60b6216474b74a14af6abf977e312b77 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 60b6216474b74a14af6abf977e312b77
I20260618 13:59:46.399299 31075 leader_election.cc:290] T d8a71e334ed1400d97c965b49b677a81 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.399829 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a71e334ed1400d97c965b49b677a81" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:46.400288 28538 raft_consensus.cc:3060] T c25fa765ec794c34b0aa43761961a8c9 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.401368 28543 raft_consensus.cc:3060] T 038ea0e9c0b04db8b7ec56b065e3cc51 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.402906 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7c6c17ba06543219259749d7d38832d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.404173 28544 raft_consensus.cc:3060] T a662bb37fb7d4df49e61e524808b67a8 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:46.408812 28618 leader_election.cc:343] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:46.410228 28618 leader_election.cc:343] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:46.411360 28618 leader_election.cc:343] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.412387 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c25fa765ec794c34b0aa43761961a8c9" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:46.413143 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a71e334ed1400d97c965b49b677a81" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:46.413537 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8e487fd51f4aa18bf59e5f8e011dcd" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:46.414122 28484 leader_election.cc:304] T c25fa765ec794c34b0aa43761961a8c9 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:46.415489 30552 raft_consensus.cc:2749] T c25fa765ec794c34b0aa43761961a8c9 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:46.416574 28484 leader_election.cc:343] T 60b6216474b74a14af6abf977e312b77 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: 60b6216474b74a14af6abf977e312b77
I20260618 13:59:46.416939 28797 tablet_bootstrap.cc:492] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:46.417070 28797 ts_tablet_manager.cc:1403] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.075s user 0.003s sys 0.003s
I20260618 13:59:46.417078 28484 leader_election.cc:304] T 60b6216474b74a14af6abf977e312b77 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:46.417452 30552 raft_consensus.cc:2749] T 60b6216474b74a14af6abf977e312b77 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:46.417924 30890 raft_consensus.cc:515] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.418180 28797 raft_consensus.cc:359] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:46.418347 28797 raft_consensus.cc:385] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.418449 28797 raft_consensus.cc:740] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.418699 28797 consensus_queue.cc:260] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:46.418939 28484 leader_election.cc:304] T d8a71e334ed1400d97c965b49b677a81 P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:46.419173 30552 raft_consensus.cc:2749] T d8a71e334ed1400d97c965b49b677a81 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:46.419224 28797 ts_tablet_manager.cc:1434] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 13:59:46.419512 28797 tablet_bootstrap.cc:492] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:46.419816 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7c6c17ba06543219259749d7d38832d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:46.419981 28411 raft_consensus.cc:3060] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.421366 30950 ts_tablet_manager.cc:1929] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:46.421409 28797 tablet_bootstrap.cc:654] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.422024 30950 log.cc:1199] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/ed648dcf882340388155068d7a5a7869
I20260618 13:59:46.422768 30950 ts_tablet_manager.cc:1950] T ed648dcf882340388155068d7a5a7869 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:46.424026 28411 raft_consensus.cc:2468] T d7c6c17ba06543219259749d7d38832d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:46.419276 28484 leader_election.cc:304] T ff8e487fd51f4aa18bf59e5f8e011dcd P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:46.424350 30552 raft_consensus.cc:2749] T ff8e487fd51f4aa18bf59e5f8e011dcd P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:46.424393 28617 leader_election.cc:304] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.424674 30898 raft_consensus.cc:2804] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:46.424799 30898 raft_consensus.cc:697] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:46.425050 30898 consensus_queue.cc:237] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.425567 30986 tablet.cc:1724] T 957670718c8c4b8cb87115cc4a2fa861 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:46.426456 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet ed648dcf882340388155068d7a5a7869 (table default.Table_8c9c9a6eea174bd68d8fb9d9ebaaf603 [id=185e7792e79644a2aced2b949fca635b]) successfully deleted
I20260618 13:59:46.427196 30577 raft_consensus.cc:493] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.427350 30577 raft_consensus.cc:515] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.428046 30577 leader_election.cc:290] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.428620 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbfebfca436461ab63dfe469ae14b4c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:46.428812 28403 raft_consensus.cc:2468] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:46.429287 28617 leader_election.cc:304] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.429848 30847 raft_consensus.cc:2804] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:46.429992 30847 raft_consensus.cc:493] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:46.430121 30847 raft_consensus.cc:3060] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.433404 30847 raft_consensus.cc:515] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.438764 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbfebfca436461ab63dfe469ae14b4c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:46.438941 28411 raft_consensus.cc:3060] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.441301 30847 leader_election.cc:290] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.441847 28411 raft_consensus.cc:2468] T 5cbfebfca436461ab63dfe469ae14b4c P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:46.442610 28617 leader_election.cc:304] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.442893 30856 raft_consensus.cc:2804] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:46.443272 30856 raft_consensus.cc:697] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:46.443704 28798 tablet_bootstrap.cc:492] T e196dd3c7fe14524a50152643ddbc2ad P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:46.443849 28798 ts_tablet_manager.cc:1403] T e196dd3c7fe14524a50152643ddbc2ad P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.127s user 0.001s sys 0.005s
I20260618 13:59:46.443754 30856 consensus_queue.cc:237] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.444511 28798 raft_consensus.cc:359] T e196dd3c7fe14524a50152643ddbc2ad P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.444697 28798 raft_consensus.cc:385] T e196dd3c7fe14524a50152643ddbc2ad P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.444764 28798 raft_consensus.cc:740] T e196dd3c7fe14524a50152643ddbc2ad P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.444937 28798 consensus_queue.cc:260] T e196dd3c7fe14524a50152643ddbc2ad P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.445744 28798 ts_tablet_manager.cc:1434] T e196dd3c7fe14524a50152643ddbc2ad P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:46.445780 30890 leader_election.cc:290] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.446193 28798 tablet_bootstrap.cc:492] T ac9f580e35af4d4080d313b0283c0b7e P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:46.448112 31048 tablet_bootstrap.cc:654] T 105f61bdf2d541cab9423b6df285265e P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.448835 28798 tablet_bootstrap.cc:654] T ac9f580e35af4d4080d313b0283c0b7e P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.449260 30634 server_base.cc:1061] running on GCE node
I20260618 13:59:46.450933 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbfebfca436461ab63dfe469ae14b4c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:46.453658 31092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 13:59:46.454771 30634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 13:59:46.457473 30950 tablet_replica.cc:333] T 39b06ae7444a48dbbec9e096fd7e4052 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:46.457695 30950 raft_consensus.cc:2243] T 39b06ae7444a48dbbec9e096fd7e4052 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:46.457858 30950 raft_consensus.cc:2272] T 39b06ae7444a48dbbec9e096fd7e4052 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:46.458362 30950 ts_tablet_manager.cc:1916] T 39b06ae7444a48dbbec9e096fd7e4052 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:46.458592 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbfebfca436461ab63dfe469ae14b4c" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.459410 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7c6c17ba06543219259749d7d38832d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:46.460826 28618 leader_election.cc:343] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:46.461194 28618 leader_election.cc:343] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.462677 30890 raft_consensus.cc:493] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.463097 28797 tablet_bootstrap.cc:492] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:46.463279 28797 ts_tablet_manager.cc:1403] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.044s user 0.006s sys 0.012s
I20260618 13:59:46.463096 30890 raft_consensus.cc:515] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.464171 30890 leader_election.cc:290] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:46.464592 28618 leader_election.cc:343] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:46.464592 28617 leader_election.cc:343] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3
I20260618 13:59:46.464074 28797 raft_consensus.cc:359] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.464854 28797 raft_consensus.cc:385] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.464927 28797 raft_consensus.cc:740] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
W20260618 13:59:46.465034 28346 consensus_peers.cc:597] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:46.465152 28797 consensus_queue.cc:260] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.465389 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "240c1f7ac1ee4fcb9e48dc3d4bbc4dd3" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.465612 28797 ts_tablet_manager.cc:1434] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:46.465901 28797 tablet_bootstrap.cc:492] T 4cb78c3ca3b74b399d2b11320473e463 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:46.466694 28618 leader_election.cc:343] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.466830 28618 leader_election.cc:304] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a169b16ced46ca9cfbc631709bb257; no voters: 6fc808af0e884ed8836561665d8e0e7b, 97859748af2c41879ba2a668dda999d2
I20260618 13:59:46.468057 31048 tablet_bootstrap.cc:492] T 105f61bdf2d541cab9423b6df285265e P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:46.468182 31048 ts_tablet_manager.cc:1403] T 105f61bdf2d541cab9423b6df285265e P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.127s user 0.000s sys 0.003s
I20260618 13:59:46.468832 31048 raft_consensus.cc:359] T 105f61bdf2d541cab9423b6df285265e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.468999 31048 raft_consensus.cc:385] T 105f61bdf2d541cab9423b6df285265e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.469043 31048 raft_consensus.cc:740] T 105f61bdf2d541cab9423b6df285265e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.469574 30898 raft_consensus.cc:2749] T 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:46.470614 31048 consensus_queue.cc:260] T 105f61bdf2d541cab9423b6df285265e P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.474092 28798 tablet_bootstrap.cc:492] T ac9f580e35af4d4080d313b0283c0b7e P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:46.474223 28798 ts_tablet_manager.cc:1403] T ac9f580e35af4d4080d313b0283c0b7e P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.003s
I20260618 13:59:46.474864 28798 raft_consensus.cc:359] T ac9f580e35af4d4080d313b0283c0b7e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.475301 28798 raft_consensus.cc:385] T ac9f580e35af4d4080d313b0283c0b7e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.475369 28798 raft_consensus.cc:740] T ac9f580e35af4d4080d313b0283c0b7e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.475565 28798 consensus_queue.cc:260] T ac9f580e35af4d4080d313b0283c0b7e P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.477064 31048 ts_tablet_manager.cc:1434] T 105f61bdf2d541cab9423b6df285265e P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260618 13:59:46.477710 30890 raft_consensus.cc:493] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.477840 30890 raft_consensus.cc:515] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.478953 30890 leader_election.cc:290] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.479681 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df0bb1a20043049275fdc25084dc0d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:46.479863 28403 raft_consensus.cc:2468] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:46.480115 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df0bb1a20043049275fdc25084dc0d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.480316 28798 ts_tablet_manager.cc:1434] T ac9f580e35af4d4080d313b0283c0b7e P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260618 13:59:46.480302 28617 leader_election.cc:304] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.480847 30856 raft_consensus.cc:2804] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:46.481285 30856 raft_consensus.cc:493] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:46.481472 30856 raft_consensus.cc:3060] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.483565 28798 tablet_bootstrap.cc:492] T c5836982752b4209b72a372c1dfeb71c P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:46.484402 30856 raft_consensus.cc:515] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.484858 28798 tablet_bootstrap.cc:654] T c5836982752b4209b72a372c1dfeb71c P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.485275 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df0bb1a20043049275fdc25084dc0d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:46.485441 28411 raft_consensus.cc:3060] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.488837 28411 raft_consensus.cc:2468] T 60df0bb1a20043049275fdc25084dc0d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:46.489226 28617 leader_election.cc:304] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.489492 30898 raft_consensus.cc:2804] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:46.489596 30898 raft_consensus.cc:697] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:46.489797 30898 consensus_queue.cc:237] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.493144 30856 leader_election.cc:290] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.494012 30634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
W20260618 13:59:46.481280 28618 leader_election.cc:343] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.496990 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60df0bb1a20043049275fdc25084dc0d" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:46.498551 28618 leader_election.cc:343] T 60df0bb1a20043049275fdc25084dc0d P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:46.498823 28618 consensus_peers.cc:597] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:46.499670 30950 ts_tablet_manager.cc:1929] T 39b06ae7444a48dbbec9e096fd7e4052 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:46.499826 30950 log.cc:1199] T 39b06ae7444a48dbbec9e096fd7e4052 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/39b06ae7444a48dbbec9e096fd7e4052
I20260618 13:59:46.498839 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:46.500416 30950 ts_tablet_manager.cc:1950] T 39b06ae7444a48dbbec9e096fd7e4052 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:46.501109 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:46.502197 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 39b06ae7444a48dbbec9e096fd7e4052 (table default.Table_a244f6a1e71a440dbc5d8a2b3a17f61a [id=1b34fcbd96004d8c9084c7360106aa15]) successfully deleted
I20260618 13:59:46.503378 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:46.503551 30634 hybrid_clock.cc:648] HybridClock initialized: now 1781791186503459 us; error 78 us; skew 500 ppm
I20260618 13:59:46.503856 28798 tablet_bootstrap.cc:492] T c5836982752b4209b72a372c1dfeb71c P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:46.503990 28798 ts_tablet_manager.cc:1403] T c5836982752b4209b72a372c1dfeb71c P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20260618 13:59:46.504020 30634 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 13:59:46.504588 28798 raft_consensus.cc:359] T c5836982752b4209b72a372c1dfeb71c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.504755 28798 raft_consensus.cc:385] T c5836982752b4209b72a372c1dfeb71c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.504803 28798 raft_consensus.cc:740] T c5836982752b4209b72a372c1dfeb71c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.504987 28798 consensus_queue.cc:260] T c5836982752b4209b72a372c1dfeb71c P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.506317 28797 tablet_bootstrap.cc:654] T 4cb78c3ca3b74b399d2b11320473e463 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.506839 31048 tablet_bootstrap.cc:492] T 931911d377ea4e3bbecd9346e081776a P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:46.508168 30552 raft_consensus.cc:493] T ac9f580e35af4d4080d313b0283c0b7e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.508297 30552 raft_consensus.cc:515] T ac9f580e35af4d4080d313b0283c0b7e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.508852 30552 leader_election.cc:290] T ac9f580e35af4d4080d313b0283c0b7e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.509534 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac9f580e35af4d4080d313b0283c0b7e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:46.510411 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac9f580e35af4d4080d313b0283c0b7e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:46.510535 28798 ts_tablet_manager.cc:1434] T c5836982752b4209b72a372c1dfeb71c P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260618 13:59:46.511193 28798 tablet_bootstrap.cc:492] T b2a6fabad00642528913e735929e295e P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:46.511499 28484 leader_election.cc:304] T ac9f580e35af4d4080d313b0283c0b7e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:46.511775 30552 raft_consensus.cc:2749] T ac9f580e35af4d4080d313b0283c0b7e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:46.513460 28798 tablet_bootstrap.cc:654] T b2a6fabad00642528913e735929e295e P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.514411 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:46.514925 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_3e7ca0884351432197632927d9c9db55 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 2 got 1
I20260618 13:59:46.515270 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:46.517388 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:46.517678 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:46.522121 28797 tablet_bootstrap.cc:492] T 4cb78c3ca3b74b399d2b11320473e463 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:46.522244 28797 ts_tablet_manager.cc:1403] T 4cb78c3ca3b74b399d2b11320473e463 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.056s user 0.017s sys 0.005s
I20260618 13:59:46.522220 28543 raft_consensus.cc:3060] T db62b3a1e37b447c9ea3fbd65c0cf1e7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.522845 28797 raft_consensus.cc:359] T 4cb78c3ca3b74b399d2b11320473e463 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.523002 28797 raft_consensus.cc:385] T 4cb78c3ca3b74b399d2b11320473e463 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.523051 28797 raft_consensus.cc:740] T 4cb78c3ca3b74b399d2b11320473e463 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.523208 28797 consensus_queue.cc:260] T 4cb78c3ca3b74b399d2b11320473e463 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.523702 28797 ts_tablet_manager.cc:1434] T 4cb78c3ca3b74b399d2b11320473e463 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 13:59:46.524381 28797 tablet_bootstrap.cc:492] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:46.525708 28797 tablet_bootstrap.cc:654] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.526829 31048 tablet_bootstrap.cc:654] T 931911d377ea4e3bbecd9346e081776a P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.518008 29011 catalog_manager.cc:5696] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:46.530262 29011 catalog_manager.cc:5696] T 779755a607bf46219900412b527e58b5 P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:46.531355 28798 tablet_bootstrap.cc:492] T b2a6fabad00642528913e735929e295e P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:46.531517 28798 ts_tablet_manager.cc:1403] T b2a6fabad00642528913e735929e295e P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20260618 13:59:46.531927 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:46.532223 28798 raft_consensus.cc:359] T b2a6fabad00642528913e735929e295e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.532411 28798 raft_consensus.cc:385] T b2a6fabad00642528913e735929e295e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.532462 28798 raft_consensus.cc:740] T b2a6fabad00642528913e735929e295e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.532717 28798 consensus_queue.cc:260] T b2a6fabad00642528913e735929e295e P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.533428 28798 ts_tablet_manager.cc:1434] T b2a6fabad00642528913e735929e295e P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260618 13:59:46.533692 28798 tablet_bootstrap.cc:492] T 0e6f3f8431cd49b5bf1306bc7609761d P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:46.536242 31048 tablet_bootstrap.cc:492] T 931911d377ea4e3bbecd9346e081776a P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:46.536372 31048 ts_tablet_manager.cc:1403] T 931911d377ea4e3bbecd9346e081776a P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.001s
I20260618 13:59:46.536957 28798 tablet_bootstrap.cc:654] T 0e6f3f8431cd49b5bf1306bc7609761d P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.536993 31048 raft_consensus.cc:359] T 931911d377ea4e3bbecd9346e081776a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.537140 31048 raft_consensus.cc:385] T 931911d377ea4e3bbecd9346e081776a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.537182 31048 raft_consensus.cc:740] T 931911d377ea4e3bbecd9346e081776a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.537906 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:46.538813 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:46.542727 31056 raft_consensus.cc:493] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.542845 31056 raft_consensus.cc:515] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.542776 30856 raft_consensus.cc:493] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.543404 31056 leader_election.cc:290] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:46.543370 30856 raft_consensus.cc:515] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.544428 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6ba4976bb6e4c2da3a563ccad4eb2b0" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.544430 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6ba4976bb6e4c2da3a563ccad4eb2b0" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:46.544586 28682 raft_consensus.cc:2468] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:46.545245 28346 leader_election.cc:304] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.545406 30856 leader_election.cc:290] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:46.545580 28346 leader_election.cc:343] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.545693 28411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00a2abfda3ef40eca1f30ff7a1ec050e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:46.545877 28411 raft_consensus.cc:2468] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:46.546006 30262 raft_consensus.cc:2804] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:46.546108 30262 raft_consensus.cc:493] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:46.546188 30262 raft_consensus.cc:3060] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.546275 28617 leader_election.cc:304] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.546653 30890 raft_consensus.cc:2804] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:46.546729 30890 raft_consensus.cc:493] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:46.546819 30890 raft_consensus.cc:3060] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.548964 30262 raft_consensus.cc:515] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.549449 30262 leader_election.cc:290] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:46.550381 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6ba4976bb6e4c2da3a563ccad4eb2b0" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:46.550359 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6ba4976bb6e4c2da3a563ccad4eb2b0" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:46.550545 28682 raft_consensus.cc:3060] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.550648 30890 raft_consensus.cc:515] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:46.551000 28346 leader_election.cc:343] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.551687 30890 leader_election.cc:290] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.551748 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00a2abfda3ef40eca1f30ff7a1ec050e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:46.551887 28403 raft_consensus.cc:3060] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.553251 28682 raft_consensus.cc:2468] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:46.553745 28346 leader_election.cc:304] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:46.554075 30262 raft_consensus.cc:2804] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:46.554186 30262 raft_consensus.cc:697] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:46.554270 28403 raft_consensus.cc:2468] T 00a2abfda3ef40eca1f30ff7a1ec050e P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:46.554550 28546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00a2abfda3ef40eca1f30ff7a1ec050e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.554935 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00a2abfda3ef40eca1f30ff7a1ec050e" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:46.555074 28617 leader_election.cc:304] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.554391 30262 consensus_queue.cc:237] T a6ba4976bb6e4c2da3a563ccad4eb2b0 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.555392 30890 raft_consensus.cc:2804] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:46.555742 30890 raft_consensus.cc:697] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:46.556054 30890 consensus_queue.cc:237] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.557399 31056 consensus_queue.cc:1048] T 5ffd607dd9eb43e0a3769fcdd28ed05f P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:46.558552 28618 leader_election.cc:343] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.559684 30634 webserver.cc:533] Webserver started at http://127.27.100.60:38293/ using document root <none> and password file <none>
I20260618 13:59:46.560976 28797 tablet_bootstrap.cc:492] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:46.561110 28797 ts_tablet_manager.cc:1403] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.037s user 0.007s sys 0.005s
W20260618 13:59:46.561364 28618 leader_election.cc:343] T 00a2abfda3ef40eca1f30ff7a1ec050e P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.561779 28797 raft_consensus.cc:359] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.561940 28797 raft_consensus.cc:385] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.562006 28797 raft_consensus.cc:740] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.539974 29011 catalog_manager.cc:5696] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1). New cstate: current_term: 1 leader_uuid: "6fc808af0e884ed8836561665d8e0e7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:46.563685 30634 fs_manager.cc:362] Metadata directory not provided
I20260618 13:59:46.564109 30634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 13:59:46.564148 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:46.566978 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet e196dd3c7fe14524a50152643ddbc2ad (table default.Table_2dfebd62a146446f999f1a7a1bd1af79 [id=02ffed3d852845d6845fb5176f036fce]). Expected version 1 got 0
I20260618 13:59:46.567011 28797 consensus_queue.cc:260] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.567602 28797 ts_tablet_manager.cc:1434] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260618 13:59:46.570518 31048 consensus_queue.cc:260] T 931911d377ea4e3bbecd9346e081776a P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.571005 31048 ts_tablet_manager.cc:1434] T 931911d377ea4e3bbecd9346e081776a P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.035s user 0.002s sys 0.000s
I20260618 13:59:46.571256 31048 tablet_bootstrap.cc:492] T f683917aac5945ec939664ab82c24530 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:46.573722 28797 tablet_bootstrap.cc:492] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:46.575343 28797 tablet_bootstrap.cc:654] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.577696 29011 catalog_manager.cc:5788] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675) does not have the latest schema for tablet 00b602a473a94c05a617fb9478300c74 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:46.585084 28798 tablet_bootstrap.cc:492] T 0e6f3f8431cd49b5bf1306bc7609761d P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:46.585243 28798 ts_tablet_manager.cc:1403] T 0e6f3f8431cd49b5bf1306bc7609761d P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.052s user 0.008s sys 0.000s
I20260618 13:59:46.586095 28798 raft_consensus.cc:359] T 0e6f3f8431cd49b5bf1306bc7609761d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.586273 28798 raft_consensus.cc:385] T 0e6f3f8431cd49b5bf1306bc7609761d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.586334 28798 raft_consensus.cc:740] T 0e6f3f8431cd49b5bf1306bc7609761d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.586578 28798 consensus_queue.cc:260] T 0e6f3f8431cd49b5bf1306bc7609761d P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.586732 28797 tablet_bootstrap.cc:492] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:46.586848 28797 ts_tablet_manager.cc:1403] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260618 13:59:46.587530 28797 raft_consensus.cc:359] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.587731 28797 raft_consensus.cc:385] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.587781 28797 raft_consensus.cc:740] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.587975 28797 consensus_queue.cc:260] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.588624 28797 ts_tablet_manager.cc:1434] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:46.589170 28797 tablet_bootstrap.cc:492] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:46.590739 28797 tablet_bootstrap.cc:654] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.592787 31048 tablet_bootstrap.cc:654] T f683917aac5945ec939664ab82c24530 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.599957 30634 fs_manager.cc:714] Time spent opening directory manager: real 0.020s user 0.004s sys 0.000s
I20260618 13:59:46.602653 29007 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet dbc88bf849a94cb9bd1c9c14aec53eb1
I20260618 13:59:46.603255 29007 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 154) took 3349 ms. Trace:
I20260618 13:59:46.603367 29007 rpcz_store.cc:276] 0618 13:59:43.253955 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:45.270859 (+2016904us) service_pool.cc:224] Handling call
0618 13:59:45.286898 (+ 16039us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb46856b6 00007fcbab0e2486 00007fcbab0e21b5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:45.287218 (+ 320us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:45.287335 (+ 117us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:45.287469 (+ 134us) catalog_manager.cc:2849] Locking tablets
0618 13:59:45.480385 (+192916us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:45.489074 (+ 8689us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:45.489152 (+ 78us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:46.602578 (+1113426us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:46.603216 (+ 638us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:45.481989 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:45.482107 (+ 118us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:45.482114 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:45.482117 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:45.482224 (+ 107us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:45.482236 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:45.482242 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:45.482374 (+ 132us) tablet.cc:701] Row locks acquired
0618 13:59:45.482378 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:45.482412 (+ 34us) write_op.cc:270] Start()
0618 13:59:45.482464 (+ 52us) write_op.cc:276] Timestamp: P: 1781791185482390 usec, L: 0
0618 13:59:45.482469 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:45.482675 (+ 206us) log.cc:844] Serialized 728 byte log entry
0618 13:59:45.487494 (+ 4819us) op_driver.cc:464] REPLICATION: finished
0618 13:59:45.488249 (+ 755us) write_op.cc:301] APPLY: starting
0618 13:59:45.488258 (+ 9us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:45.488286 (+ 28us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:45.488307 (+ 21us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:45.488364 (+ 57us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:45.488390 (+ 26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:45.488396 (+ 6us) write_op.cc:312] APPLY: finished
0618 13:59:45.488453 (+ 57us) log.cc:844] Serialized 29 byte log entry
0618 13:59:45.488494 (+ 41us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.488511 (+ 17us) write_op.cc:454] Released schema lock
0618 13:59:45.488517 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:45.545183 (+ 56666us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.545195 (+ 12us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":36396416,"child_traces":[["op",{"apply.queue_time_us":724,"apply.run_cpu_time_us":334,"apply.run_wall_time_us":333,"num_ops":2,"prepare.queue_time_us":928,"prepare.run_cpu_time_us":8403,"prepare.run_wall_time_us":63161,"raft.queue_time_us":1265,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":7,"replication_time_us":5005,"thread_start_us":55109,"threads_started":1}]]}
I20260618 13:59:46.604894 28797 tablet_bootstrap.cc:492] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:46.605058 28797 ts_tablet_manager.cc:1403] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.000s
I20260618 13:59:46.605739 28797 raft_consensus.cc:359] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.605903 28797 raft_consensus.cc:385] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.605955 28797 raft_consensus.cc:740] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.606169 28797 consensus_queue.cc:260] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.606671 28797 ts_tablet_manager.cc:1434] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:46.606930 28797 tablet_bootstrap.cc:492] T 66ada05b01b2427da182998a557529ca P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:46.607273 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet dbc88bf849a94cb9bd1c9c14aec53eb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:45 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:46.608368 28651 tablet_service.cc:1558] Processing DeleteTablet for tablet dbc88bf849a94cb9bd1c9c14aec53eb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:45 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:46.608570 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet dbc88bf849a94cb9bd1c9c14aec53eb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:45 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:46.608731 29566 tablet_replica.cc:333] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:46.608891 29566 raft_consensus.cc:2243] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:46.609067 29566 pending_rounds.cc:70] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257: Trying to abort 1 pending ops.
I20260618 13:59:46.609113 29566 pending_rounds.cc:77] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298216699436265472 op_type: NO_OP noop_request { }
I20260618 13:59:46.609182 29566 raft_consensus.cc:2889] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260618 13:59:46.609261 29566 raft_consensus.cc:2272] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:46.609898 28797 tablet_bootstrap.cc:654] T 66ada05b01b2427da182998a557529ca P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.610740 28798 ts_tablet_manager.cc:1434] T 0e6f3f8431cd49b5bf1306bc7609761d P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.025s user 0.002s sys 0.000s
W20260618 13:59:46.610780 28989 catalog_manager.cc:5007] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): delete failed for tablet dbc88bf849a94cb9bd1c9c14aec53eb1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dbc88bf849a94cb9bd1c9c14aec53eb1 already in progress: creating tablet
I20260618 13:59:46.611682 29566 ts_tablet_manager.cc:1916] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:46.617029 28797 tablet_bootstrap.cc:492] T 66ada05b01b2427da182998a557529ca P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:46.617151 28797 ts_tablet_manager.cc:1403] T 66ada05b01b2427da182998a557529ca P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.001s
I20260618 13:59:46.617821 28797 raft_consensus.cc:359] T 66ada05b01b2427da182998a557529ca P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.617985 28797 raft_consensus.cc:385] T 66ada05b01b2427da182998a557529ca P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.618034 28797 raft_consensus.cc:740] T 66ada05b01b2427da182998a557529ca P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.618103 28798 tablet_bootstrap.cc:492] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:46.618222 28797 consensus_queue.cc:260] T 66ada05b01b2427da182998a557529ca P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.618863 28797 ts_tablet_manager.cc:1434] T 66ada05b01b2427da182998a557529ca P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 13:59:46.619114 28797 tablet_bootstrap.cc:492] T 134a43259d5b429fbe47d0555d9942f3 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:46.619722 28798 tablet_bootstrap.cc:654] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.620690 29566 ts_tablet_manager.cc:1929] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:46.622535 29566 log.cc:1199] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/dbc88bf849a94cb9bd1c9c14aec53eb1
I20260618 13:59:46.622983 29566 ts_tablet_manager.cc:1950] T dbc88bf849a94cb9bd1c9c14aec53eb1 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:46.624105 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet dbc88bf849a94cb9bd1c9c14aec53eb1 (table default.Table_bb918931281942b486860c5e4f5755f1 [id=74cfa445524642d9a804da39d46b6b32]) successfully deleted
I20260618 13:59:46.638402 30939 tablet_replica.cc:333] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:46.638698 30939 raft_consensus.cc:2243] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:46.638865 30939 raft_consensus.cc:2272] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:46.639446 30939 ts_tablet_manager.cc:1916] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:46.621598 28797 tablet_bootstrap.cc:654] T 134a43259d5b429fbe47d0555d9942f3 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.641711 31127 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 13:59:46.642030 30890 consensus_queue.cc:1048] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:46.644542 28798 tablet_bootstrap.cc:492] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:46.644770 28798 ts_tablet_manager.cc:1403] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.027s user 0.003s sys 0.000s
I20260618 13:59:46.645633 28798 raft_consensus.cc:359] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.653486 28375 tablet_service.cc:1511] Processing CreateTablet for tablet abb71ad1e9954e9e9551a9fa6b883e19 (DEFAULT_TABLE table=default.Table_0b63e77c05484ae28bd843613ed1b0cb [id=b9801f39dcd647fab9732aeec1d2f768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:46.653867 28375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abb71ad1e9954e9e9551a9fa6b883e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:46.653972 28651 tablet_service.cc:1511] Processing CreateTablet for tablet abb71ad1e9954e9e9551a9fa6b883e19 (DEFAULT_TABLE table=default.Table_0b63e77c05484ae28bd843613ed1b0cb [id=b9801f39dcd647fab9732aeec1d2f768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:46.654237 30634 fs_manager.cc:730] Time spent opening block manager: real 0.052s user 0.006s sys 0.006s
I20260618 13:59:46.654317 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abb71ad1e9954e9e9551a9fa6b883e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:46.654467 30634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data,/tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/wal
uuid: "7637285570574a6dbcef1de6f84ecac0"
format_stamp: "Formatted at 2026-06-18 13:59:16 on dist-test-slave-bmq2"
I20260618 13:59:46.655889 30634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 13:59:46.665701 28797 tablet_bootstrap.cc:492] T 134a43259d5b429fbe47d0555d9942f3 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:46.665843 28797 ts_tablet_manager.cc:1403] T 134a43259d5b429fbe47d0555d9942f3 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.002s
I20260618 13:59:46.666610 28797 raft_consensus.cc:359] T 134a43259d5b429fbe47d0555d9942f3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.666772 28797 raft_consensus.cc:385] T 134a43259d5b429fbe47d0555d9942f3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.666826 28797 raft_consensus.cc:740] T 134a43259d5b429fbe47d0555d9942f3 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.667487 28797 consensus_queue.cc:260] T 134a43259d5b429fbe47d0555d9942f3 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:46.668378 28618 consensus_peers.cc:597] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:46.673254 30939 ts_tablet_manager.cc:1929] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260618 13:59:46.673578 30939 log.cc:1199] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/dbc88bf849a94cb9bd1c9c14aec53eb1
I20260618 13:59:46.674067 30939 ts_tablet_manager.cc:1950] T dbc88bf849a94cb9bd1c9c14aec53eb1 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:46.674796 28797 ts_tablet_manager.cc:1434] T 134a43259d5b429fbe47d0555d9942f3 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260618 13:59:46.676126 28797 tablet_bootstrap.cc:492] T 6827859dd7be4464a9d98ce502df8539 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:46.677951 28797 tablet_bootstrap.cc:654] T 6827859dd7be4464a9d98ce502df8539 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.681416 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet dbc88bf849a94cb9bd1c9c14aec53eb1 (table default.Table_bb918931281942b486860c5e4f5755f1 [id=74cfa445524642d9a804da39d46b6b32]) successfully deleted
I20260618 13:59:46.686532 31048 tablet_bootstrap.cc:492] T f683917aac5945ec939664ab82c24530 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:46.686705 31048 ts_tablet_manager.cc:1403] T f683917aac5945ec939664ab82c24530 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.116s user 0.001s sys 0.017s
I20260618 13:59:46.690099 29007 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:46.659133 28517 tablet_service.cc:1511] Processing CreateTablet for tablet abb71ad1e9954e9e9551a9fa6b883e19 (DEFAULT_TABLE table=default.Table_0b63e77c05484ae28bd843613ed1b0cb [id=b9801f39dcd647fab9732aeec1d2f768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:46.690508 28798 raft_consensus.cc:385] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.690603 28798 raft_consensus.cc:740] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.690800 28517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abb71ad1e9954e9e9551a9fa6b883e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:46.673470 28543 raft_consensus.cc:3060] T ac9f580e35af4d4080d313b0283c0b7e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.690815 28798 consensus_queue.cc:260] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.698526 28798 ts_tablet_manager.cc:1434] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.054s user 0.002s sys 0.000s
I20260618 13:59:46.698827 28798 tablet_bootstrap.cc:492] T 52006500f117417b8a2c979d4aeb077c P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:46.701699 31048 raft_consensus.cc:359] T f683917aac5945ec939664ab82c24530 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:46.701691 30856 consensus_queue.cc:1048] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:46.701861 31048 raft_consensus.cc:385] T f683917aac5945ec939664ab82c24530 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.701917 31048 raft_consensus.cc:740] T f683917aac5945ec939664ab82c24530 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.703478 28797 tablet_bootstrap.cc:492] T 6827859dd7be4464a9d98ce502df8539 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:46.703632 28797 ts_tablet_manager.cc:1403] T 6827859dd7be4464a9d98ce502df8539 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.028s user 0.004s sys 0.000s
I20260618 13:59:46.704454 28797 raft_consensus.cc:359] T 6827859dd7be4464a9d98ce502df8539 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.704618 28797 raft_consensus.cc:385] T 6827859dd7be4464a9d98ce502df8539 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.704697 28797 raft_consensus.cc:740] T 6827859dd7be4464a9d98ce502df8539 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.704893 28797 consensus_queue.cc:260] T 6827859dd7be4464a9d98ce502df8539 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.705199 30890 raft_consensus.cc:493] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.705312 30890 raft_consensus.cc:515] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:46.705848 30890 leader_election.cc:290] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.706815 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76f83f23f22345bb9bc3f239642ea673" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:46.706984 28403 raft_consensus.cc:2468] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:46.707551 28617 leader_election.cc:304] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.707764 30856 raft_consensus.cc:2804] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:46.707839 30856 raft_consensus.cc:493] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:46.707841 30864 raft_consensus.cc:493] T 105f61bdf2d541cab9423b6df285265e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.707901 30856 raft_consensus.cc:3060] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.707952 30864 raft_consensus.cc:515] T 105f61bdf2d541cab9423b6df285265e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.708169 31048 consensus_queue.cc:260] T f683917aac5945ec939664ab82c24530 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:46.708439 30864 leader_election.cc:290] T 105f61bdf2d541cab9423b6df285265e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:46.708613 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76f83f23f22345bb9bc3f239642ea673" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.708825 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "105f61bdf2d541cab9423b6df285265e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:46.709235 28618 leader_election.cc:343] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.709689 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "105f61bdf2d541cab9423b6df285265e" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:46.710583 28346 leader_election.cc:343] T 105f61bdf2d541cab9423b6df285265e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:46.710769 28346 leader_election.cc:343] T 105f61bdf2d541cab9423b6df285265e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.710848 28346 leader_election.cc:304] T 105f61bdf2d541cab9423b6df285265e P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:46.710340 30856 raft_consensus.cc:515] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:46.712646 30856 leader_election.cc:290] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.712976 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76f83f23f22345bb9bc3f239642ea673" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:46.713523 28618 leader_election.cc:343] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.714164 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76f83f23f22345bb9bc3f239642ea673" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:46.714359 28403 raft_consensus.cc:3060] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.714609 29007 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:46.715752 29007 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:46.716388 29007 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:46.716790 29007 catalog_manager.cc:5696] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:46.717831 28403 raft_consensus.cc:2468] T 76f83f23f22345bb9bc3f239642ea673 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:46.717828 29007 catalog_manager.cc:5696] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:46.718863 29007 catalog_manager.cc:5696] T 08132883fb4f4632af2961bead244b18 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260618 13:59:46.718998 28617 leader_election.cc:304] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:46.719417 30847 raft_consensus.cc:2804] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:46.719553 30847 raft_consensus.cc:697] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:46.719573 29007 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:46.719782 30847 consensus_queue.cc:237] T 76f83f23f22345bb9bc3f239642ea673 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:46.720067 29007 catalog_manager.cc:5696] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:46.720912 29007 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet e196dd3c7fe14524a50152643ddbc2ad (table default.Table_2dfebd62a146446f999f1a7a1bd1af79 [id=02ffed3d852845d6845fb5176f036fce]). Expected version 1 got 0
I20260618 13:59:46.721643 29007 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet a1896e3b4a904fef8a741f63aa952251 (table default.Table_21242eba3cb64c7c91082e296439cfe2 [id=10a4f5ef579746b68d8dd37d83f14c3e]). Expected version 1 got 0
I20260618 13:59:46.721698 28797 ts_tablet_manager.cc:1434] T 6827859dd7be4464a9d98ce502df8539 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.018s user 0.002s sys 0.000s
I20260618 13:59:46.722467 29007 catalog_manager.cc:5696] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:46.723150 29007 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet f683917aac5945ec939664ab82c24530 (table default.Table_498fb462955c4472ae5aecef39e016cd [id=dcd71daa60ee430d9ca46a1463d9a1bf]). Expected version 1 got 0
I20260618 13:59:46.723448 29007 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet 00b602a473a94c05a617fb9478300c74 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:46.723903 29007 catalog_manager.cc:5696] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:46.724514 29007 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:46.724722 29007 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:46.725250 29007 catalog_manager.cc:5696] T f32d411b71d146059d40260e54c283ec P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:46.725759 29007 catalog_manager.cc:5696] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:46.726863 29007 catalog_manager.cc:5696] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:46.727018 30577 consensus_queue.cc:1048] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:46.727216 28797 tablet_bootstrap.cc:492] T 00b602a473a94c05a617fb9478300c74 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:46.728863 29007 catalog_manager.cc:5696] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3). New cstate: current_term: 1 leader_uuid: "d6a169b16ced46ca9cfbc631709bb257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260618 13:59:46.729838 29007 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:46.730352 29007 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:46.730659 29007 catalog_manager.cc:5788] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:46.731515 29025 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 152) took 3477 ms. Trace:
I20260618 13:59:46.731604 29025 rpcz_store.cc:276] 0618 13:59:43.253723 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:45.204114 (+1950391us) service_pool.cc:224] Handling call
0618 13:59:45.204503 (+ 389us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:45.204716 (+ 213us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:45.204870 (+ 154us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:45.204943 (+ 73us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:45.254969 (+ 50026us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:46.731461 (+1476492us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:45.205546 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:45.205882 (+ 336us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:45.205888 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:45.205890 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:45.206073 (+ 183us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:45.206083 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:45.206174 (+ 91us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:45.206356 (+ 182us) tablet.cc:701] Row locks acquired
0618 13:59:45.206358 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:45.206390 (+ 32us) write_op.cc:270] Start()
0618 13:59:45.206448 (+ 58us) write_op.cc:276] Timestamp: P: 1781791185206347 usec, L: 0
0618 13:59:45.206452 (+ 4us) op_driver.cc:348] REPLICATION: starting
0618 13:59:45.238526 (+ 32074us) log.cc:844] Serialized 405 byte log entry
0618 13:59:45.240596 (+ 2070us) spinlock_profiling.cc:243] Waited 31.8 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fcbac75ef22 00007fcbadb40305 00007fcbadb3ee29 00007fcbadb47250 00007fcbadb4705d 0000000000aa681c 00007fcba07299ff 00007fcba072cd1b 00007fcba072cb98 0000000000aa681c
0618 13:59:45.252466 (+ 11870us) op_driver.cc:464] REPLICATION: finished
0618 13:59:45.252573 (+ 107us) write_op.cc:301] APPLY: starting
0618 13:59:45.252589 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:45.252629 (+ 40us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:45.252631 (+ 2us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:45.252696 (+ 65us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:45.252728 (+ 32us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:45.252735 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:45.252808 (+ 73us) log.cc:844] Serialized 23 byte log entry
0618 13:59:45.252896 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.252914 (+ 18us) write_op.cc:454] Released schema lock
0618 13:59:45.252922 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:45.252990 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.252993 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":58,"apply.run_cpu_time_us":408,"apply.run_wall_time_us":407,"num_ops":1,"prepare.queue_time_us":129,"prepare.run_cpu_time_us":1126,"prepare.run_wall_time_us":35086,"replication_time_us":45981,"spinlock_wait_cycles":73253632}]]}
I20260618 13:59:46.734088 31048 ts_tablet_manager.cc:1434] T f683917aac5945ec939664ab82c24530 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.047s user 0.000s sys 0.002s
I20260618 13:59:46.736665 31048 tablet_bootstrap.cc:492] T 7eb0471903fd4654af395f7e4b68eb99 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:46.736917 30864 raft_consensus.cc:2749] T 105f61bdf2d541cab9423b6df285265e P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:46.739532 30577 consensus_queue.cc:1048] T cb7f2d02ec004b9298b94307c7a38a97 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:46.740777 28797 tablet_bootstrap.cc:654] T 00b602a473a94c05a617fb9478300c74 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:46.749466 28618 consensus_peers.cc:597] T 732c8801c319400283806aac7f15917e P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:46.753036 30262 raft_consensus.cc:493] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.753160 30262 raft_consensus.cc:515] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.753652 30262 leader_election.cc:290] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:46.754308 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cb78c3ca3b74b399d2b11320473e463" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:46.754483 28682 raft_consensus.cc:2468] T 4cb78c3ca3b74b399d2b11320473e463 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:46.755167 28346 leader_election.cc:304] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:46.755465 30262 raft_consensus.cc:2804] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:46.755535 30262 raft_consensus.cc:493] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:46.755597 30262 raft_consensus.cc:3060] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.757974 30864 raft_consensus.cc:493] T e772ef1424fc4a8f8d9e3288e718e21a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.758095 30262 raft_consensus.cc:515] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.758128 30864 raft_consensus.cc:515] T e772ef1424fc4a8f8d9e3288e718e21a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.758653 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cb78c3ca3b74b399d2b11320473e463" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.758824 30864 leader_election.cc:290] T e772ef1424fc4a8f8d9e3288e718e21a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
W20260618 13:59:46.759357 28346 leader_election.cc:343] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.759882 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e772ef1424fc4a8f8d9e3288e718e21a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:46.760541 28346 leader_election.cc:343] T e772ef1424fc4a8f8d9e3288e718e21a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.760751 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e772ef1424fc4a8f8d9e3288e718e21a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.761130 28798 tablet_bootstrap.cc:654] T 52006500f117417b8a2c979d4aeb077c P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:46.761283 28346 leader_election.cc:343] T e772ef1424fc4a8f8d9e3288e718e21a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.761416 28346 leader_election.cc:304] T e772ef1424fc4a8f8d9e3288e718e21a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:46.761667 30864 raft_consensus.cc:2749] T e772ef1424fc4a8f8d9e3288e718e21a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:46.761955 30262 leader_election.cc:290] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:46.762506 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cb78c3ca3b74b399d2b11320473e463" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:46.762647 28682 raft_consensus.cc:3060] T 4cb78c3ca3b74b399d2b11320473e463 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:46.764441 28618 consensus_peers.cc:597] T ba18e879631947f88bd266681746ec82 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:46.765177 28682 raft_consensus.cc:2468] T 4cb78c3ca3b74b399d2b11320473e463 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:46.765751 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cb78c3ca3b74b399d2b11320473e463" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:46.766192 28346 leader_election.cc:343] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.766513 28346 leader_election.cc:304] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:46.766865 30262 raft_consensus.cc:2804] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:46.767021 30262 raft_consensus.cc:697] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:46.767377 30262 consensus_queue.cc:237] T 4cb78c3ca3b74b399d2b11320473e463 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.768968 30864 raft_consensus.cc:493] T 931911d377ea4e3bbecd9346e081776a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.769090 30864 raft_consensus.cc:515] T 931911d377ea4e3bbecd9346e081776a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.769666 30864 leader_election.cc:290] T 931911d377ea4e3bbecd9346e081776a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:46.769867 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "931911d377ea4e3bbecd9346e081776a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.770651 31048 tablet_bootstrap.cc:654] T 7eb0471903fd4654af395f7e4b68eb99 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.770629 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "931911d377ea4e3bbecd9346e081776a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:46.771166 28346 leader_election.cc:343] T 931911d377ea4e3bbecd9346e081776a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:46.771494 28346 leader_election.cc:343] T 931911d377ea4e3bbecd9346e081776a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.771627 28346 leader_election.cc:304] T 931911d377ea4e3bbecd9346e081776a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:46.771857 30864 raft_consensus.cc:2749] T 931911d377ea4e3bbecd9346e081776a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:46.794390 28798 tablet_bootstrap.cc:492] T 52006500f117417b8a2c979d4aeb077c P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:46.799574 31048 tablet_bootstrap.cc:492] T 7eb0471903fd4654af395f7e4b68eb99 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:46.799850 28798 ts_tablet_manager.cc:1403] T 52006500f117417b8a2c979d4aeb077c P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.101s user 0.003s sys 0.001s
I20260618 13:59:46.800961 28798 raft_consensus.cc:359] T 52006500f117417b8a2c979d4aeb077c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.801666 31048 ts_tablet_manager.cc:1403] T 7eb0471903fd4654af395f7e4b68eb99 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.065s user 0.002s sys 0.001s
I20260618 13:59:46.803719 28798 raft_consensus.cc:385] T 52006500f117417b8a2c979d4aeb077c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.804313 31048 raft_consensus.cc:359] T 7eb0471903fd4654af395f7e4b68eb99 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.805608 28798 raft_consensus.cc:740] T 52006500f117417b8a2c979d4aeb077c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.808768 28798 consensus_queue.cc:260] T 52006500f117417b8a2c979d4aeb077c P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.809489 31048 raft_consensus.cc:385] T 7eb0471903fd4654af395f7e4b68eb99 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.809715 31048 raft_consensus.cc:740] T 7eb0471903fd4654af395f7e4b68eb99 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.810763 31048 consensus_queue.cc:260] T 7eb0471903fd4654af395f7e4b68eb99 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.811731 28798 ts_tablet_manager.cc:1434] T 52006500f117417b8a2c979d4aeb077c P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20260618 13:59:46.817245 31048 ts_tablet_manager.cc:1434] T 7eb0471903fd4654af395f7e4b68eb99 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260618 13:59:46.820163 28798 tablet_bootstrap.cc:492] T 51b91f053bcc42bb829bb8180e232c8e P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:46.827664 30552 raft_consensus.cc:493] T e196dd3c7fe14524a50152643ddbc2ad P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.827823 30552 raft_consensus.cc:515] T e196dd3c7fe14524a50152643ddbc2ad P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.828445 30552 leader_election.cc:290] T e196dd3c7fe14524a50152643ddbc2ad P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.829120 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e196dd3c7fe14524a50152643ddbc2ad" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:46.830271 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e196dd3c7fe14524a50152643ddbc2ad" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:46.830807 28484 leader_election.cc:304] T e196dd3c7fe14524a50152643ddbc2ad P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:46.831236 30552 raft_consensus.cc:2749] T e196dd3c7fe14524a50152643ddbc2ad P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:46.832218 31154 tablet.cc:1724] T ff8e487fd51f4aa18bf59e5f8e011dcd P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:46.833460 31048 tablet_bootstrap.cc:492] T aa7faa3bb1b7474cb84e5f2a96af65e8 P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:46.835041 30986 tablet.cc:1724] T ff8e487fd51f4aa18bf59e5f8e011dcd P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:46.835806 28797 tablet_bootstrap.cc:492] T 00b602a473a94c05a617fb9478300c74 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:46.835937 28797 ts_tablet_manager.cc:1403] T 00b602a473a94c05a617fb9478300c74 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.110s user 0.003s sys 0.003s
I20260618 13:59:46.836614 28797 raft_consensus.cc:359] T 00b602a473a94c05a617fb9478300c74 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.836755 28797 raft_consensus.cc:385] T 00b602a473a94c05a617fb9478300c74 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:46.836796 28797 raft_consensus.cc:740] T 00b602a473a94c05a617fb9478300c74 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:46.836946 28797 consensus_queue.cc:260] T 00b602a473a94c05a617fb9478300c74 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.837867 28797 ts_tablet_manager.cc:1434] T 00b602a473a94c05a617fb9478300c74 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:46.838099 28797 tablet_bootstrap.cc:492] T 43853b8aba3444c3a21f6416f224e0d6 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:46.839412 31158 tablet.cc:1724] T ff8e487fd51f4aa18bf59e5f8e011dcd P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:46.846257 30986 tablet.cc:1724] T 56650c9e20e24a28821cca4279bfde58 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:46.861781 31162 tablet.cc:1724] T 56650c9e20e24a28821cca4279bfde58 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:46.876900 28797 tablet_bootstrap.cc:654] T 43853b8aba3444c3a21f6416f224e0d6 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.889125 31170 tablet.cc:1724] T bf5e04eb43ed4cf493c0f6f08e22baeb P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:46.890830 31158 tablet.cc:1724] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:46.891132 30898 raft_consensus.cc:493] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.891247 30898 raft_consensus.cc:515] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.891825 30898 leader_election.cc:290] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.892807 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1cf014708ea4c4085b37c6094bf8a20" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:46.892881 30986 tablet.cc:1724] T bf5e04eb43ed4cf493c0f6f08e22baeb P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260618 13:59:46.893543 28618 leader_election.cc:343] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.896785 30856 consensus_queue.cc:1048] T e438b3733c374e1bb8aae4ed9f16b4a4 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:46.888509 31154 tablet.cc:1724] T bf5e04eb43ed4cf493c0f6f08e22baeb P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:46.894279 28401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1cf014708ea4c4085b37c6094bf8a20" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:46.903297 28401 raft_consensus.cc:2468] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:46.903687 28617 leader_election.cc:304] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:46.904177 30890 raft_consensus.cc:2804] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:46.904443 30890 raft_consensus.cc:493] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:46.904953 30890 raft_consensus.cc:3060] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.902976 31162 tablet.cc:1724] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:46.923339 30634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
W20260618 13:59:46.928647 28346 consensus_peers.cc:597] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:46.924883 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 038ea0e9c0b04db8b7ec56b065e3cc51 (table default.Table_fed3fd1cd60c4b50b1c30442807161fe [id=3bb8ec262af24e49896eb550e479c88f]). Expected version 1 got 0
I20260618 13:59:46.930286 31158 tablet.cc:1724] T 56650c9e20e24a28821cca4279bfde58 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:46.930864 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 957670718c8c4b8cb87115cc4a2fa861 (table default.Table_3e7ca0884351432197632927d9c9db55 [id=5367efaf388c4ed78132d9752c341d5b]). Expected version 2 got 1
I20260618 13:59:46.940418 28798 tablet_bootstrap.cc:654] T 51b91f053bcc42bb829bb8180e232c8e P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.940944 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet fe1139c409e84070a2f496a85545e8d5 (table default.Table_af28c507a4b04f7097bd6a034351abc8 [id=c3a54561a2e7448ebb47b0234579944e]). Expected version 1 got 0
I20260618 13:59:46.941401 30634 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 13:59:46.942404 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 56650c9e20e24a28821cca4279bfde58 (table default.Table_5de92d2ec52545f9946f35cf74ca4f01 [id=b97c43b351b946c48f356b1ea1134239]). Expected version 1 got 0
I20260618 13:59:46.942552 30634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 13:59:46.944036 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet e196dd3c7fe14524a50152643ddbc2ad (table default.Table_2dfebd62a146446f999f1a7a1bd1af79 [id=02ffed3d852845d6845fb5176f036fce]). Expected version 1 got 0
I20260618 13:59:46.944553 30986 tablet.cc:1724] T 450c68da26404767b01c5e8da3f6286d P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:46.947333 31048 tablet_bootstrap.cc:654] T aa7faa3bb1b7474cb84e5f2a96af65e8 P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:46.957263 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 00b602a473a94c05a617fb9478300c74 (table default.Table_e6090de33b2147489b3af52d6ec4dc1b [id=4961f93e6ad64ec7b3bddcc62db96522]). Expected version 2 got 0
I20260618 13:59:46.957685 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet b509f940a02545019c526e45e8372c97 (table default.Table_f44be481def649db8daed0bf151a258c [id=840c933184144aebb3659d939f679f00]). Expected version 2 got 0
I20260618 13:59:46.957890 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet f54910015987484e95b295352d859315 (table default.Table_c344ac246b4c4f11a16becbd5023032a [id=2c1b442aeecf44ee8b6140cacdb37627]). Expected version 2 got 0
I20260618 13:59:46.958211 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c25fa765ec794c34b0aa43761961a8c9 (table default.Table_b92d4faf69a64f5787f610c8f1ecca22 [id=a7d18bd3a9474b92b0612b755f6124cc]). Expected version 1 got 0
I20260618 13:59:46.959614 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet d15b1b603f024f3295eb07c98991076d (table default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e]). Expected version 1 got 0
I20260618 13:59:46.964524 30890 raft_consensus.cc:515] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.966104 30890 leader_election.cc:290] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.966187 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1cf014708ea4c4085b37c6094bf8a20" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:46.966351 28403 raft_consensus.cc:3060] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.969376 28403 raft_consensus.cc:2468] T e1cf014708ea4c4085b37c6094bf8a20 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
I20260618 13:59:46.969609 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1cf014708ea4c4085b37c6094bf8a20" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:46.969903 28617 leader_election.cc:304] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
W20260618 13:59:46.970273 28618 leader_election.cc:343] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:46.970376 30898 raft_consensus.cc:2804] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:46.970506 30898 raft_consensus.cc:697] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:46.970705 30898 consensus_queue.cc:237] T e1cf014708ea4c4085b37c6094bf8a20 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.976115 30552 raft_consensus.cc:493] T c5836982752b4209b72a372c1dfeb71c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:46.976282 30552 raft_consensus.cc:515] T c5836982752b4209b72a372c1dfeb71c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:46.976990 30552 leader_election.cc:290] T c5836982752b4209b72a372c1dfeb71c P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:46.977416 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5836982752b4209b72a372c1dfeb71c" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:46.977699 28403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5836982752b4209b72a372c1dfeb71c" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:46.978798 30856 consensus_queue.cc:1048] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:46.980463 28484 leader_election.cc:304] T c5836982752b4209b72a372c1dfeb71c P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:46.980710 30552 raft_consensus.cc:2749] T c5836982752b4209b72a372c1dfeb71c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:46.993228 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet c38a2156a55e4e5eb91d101e9a61eecd (table default.Table_5c88f7a919ad4d5da30ed365c5533883 [id=2e7a199c805d4453b2137b5fa69d0b4e]). Expected version 1 got 0
I20260618 13:59:46.995030 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet bf5e04eb43ed4cf493c0f6f08e22baeb (table default.Table_f665d2e34fc24b29b021a45c8b57a700 [id=2c73a9d87a914f03b09ef422d49f7a95]). Expected version 1 got 0
I20260618 13:59:46.996368 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet aa93f065b4d64e7b85e30f0690fae541 (table default.Table_13510793ddbc477cbdd8f32a8450fb45 [id=c64d6fa77903441385590181c6633a4b]). Expected version 1 got 0
I20260618 13:59:46.997257 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet 6903cb63c5b04408b7eaecf0c0231c42 (table default.Table_599a2901aeaf4edc9cc2f967fb5fcbae [id=408b312e0fad42ae8fb0ab84abd85c0b]). Expected version 1 got 0
I20260618 13:59:46.997877 28537 raft_consensus.cc:3060] T 0e6f3f8431cd49b5bf1306bc7609761d P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:46.998510 29025 catalog_manager.cc:5788] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749) does not have the latest schema for tablet ff8e487fd51f4aa18bf59e5f8e011dcd (table default.Table_ee2255e7c482462580209cfe2d6fd9b5 [id=899506754d9140888a566b45450f2914]). Expected version 1 got 0
I20260618 13:59:47.003785 31170 tablet.cc:1724] T e196dd3c7fe14524a50152643ddbc2ad P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:47.004534 29026 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet 450c68da26404767b01c5e8da3f6286d
W20260618 13:59:47.004863 28618 consensus_peers.cc:597] T dffeeb7313714c3f9eccc228c98ae7a3 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:47.005205 30856 raft_consensus.cc:493] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:47.005209 29026 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 156) took 3751 ms. Trace:
I20260618 13:59:47.005334 30856 raft_consensus.cc:515] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.005806 29019 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 159) took 3071 ms. Trace:
I20260618 13:59:47.005899 30856 leader_election.cc:290] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:47.005345 29026 rpcz_store.cc:276] 0618 13:59:43.254104 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:46.085740 (+2831636us) service_pool.cc:224] Handling call
0618 13:59:46.086161 (+ 421us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:46.086323 (+ 162us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:46.086392 (+ 69us) catalog_manager.cc:2849] Locking tablets
0618 13:59:46.181141 (+ 94749us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:46.189674 (+ 8533us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:46.189762 (+ 88us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:47.004456 (+814694us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:47.005159 (+ 703us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:46.181783 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:46.181888 (+ 105us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:46.181892 (+ 4us) write_op.cc:435] Acquired schema lock
0618 13:59:46.181895 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:46.182003 (+ 108us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:46.182026 (+ 23us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:46.182034 (+ 8us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:46.182153 (+ 119us) tablet.cc:701] Row locks acquired
0618 13:59:46.182156 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:46.182195 (+ 39us) write_op.cc:270] Start()
0618 13:59:46.182229 (+ 34us) write_op.cc:276] Timestamp: P: 1781791186182162 usec, L: 0
0618 13:59:46.182234 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:46.182435 (+ 201us) log.cc:844] Serialized 793 byte log entry
0618 13:59:46.188808 (+ 6373us) op_driver.cc:464] REPLICATION: finished
0618 13:59:46.189166 (+ 358us) write_op.cc:301] APPLY: starting
0618 13:59:46.189181 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:46.189224 (+ 43us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:46.189259 (+ 35us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:46.189320 (+ 61us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:46.189352 (+ 32us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:46.189359 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:46.189438 (+ 79us) log.cc:844] Serialized 29 byte log entry
0618 13:59:46.189491 (+ 53us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:46.189510 (+ 19us) write_op.cc:454] Released schema lock
0618 13:59:46.189517 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:46.189579 (+ 62us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:46.189583 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":306,"apply.run_cpu_time_us":407,"apply.run_wall_time_us":407,"num_ops":2,"prepare.queue_time_us":53,"prepare.run_cpu_time_us":799,"prepare.run_wall_time_us":2644,"raft.queue_time_us":66,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":303,"replication_time_us":6534}]]}
I20260618 13:59:47.005909 29019 rpcz_store.cc:276] 0618 13:59:43.934766 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:46.146655 (+2211889us) service_pool.cc:224] Handling call
0618 13:59:46.147159 (+ 504us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:46.147423 (+ 264us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:46.147645 (+ 222us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:46.147737 (+ 92us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:46.175700 (+ 27963us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:47.005735 (+830035us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:46.148299 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:46.148414 (+ 115us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:46.148419 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:46.148421 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:46.148510 (+ 89us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:46.148522 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:46.148526 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:46.148584 (+ 58us) tablet.cc:701] Row locks acquired
0618 13:59:46.148596 (+ 12us) write_op.cc:260] PREPARE: finished
0618 13:59:46.148628 (+ 32us) write_op.cc:270] Start()
0618 13:59:46.148662 (+ 34us) write_op.cc:276] Timestamp: P: 1781791186148601 usec, L: 0
0618 13:59:46.148665 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:46.148867 (+ 202us) log.cc:844] Serialized 405 byte log entry
0618 13:59:46.163703 (+ 14836us) op_driver.cc:464] REPLICATION: finished
0618 13:59:46.163939 (+ 236us) write_op.cc:301] APPLY: starting
0618 13:59:46.163961 (+ 22us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:46.164038 (+ 77us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:46.164072 (+ 34us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:46.164131 (+ 59us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:46.164167 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:46.164177 (+ 10us) write_op.cc:312] APPLY: finished
0618 13:59:46.164258 (+ 81us) log.cc:844] Serialized 23 byte log entry
0618 13:59:46.164324 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:46.164346 (+ 22us) write_op.cc:454] Released schema lock
0618 13:59:46.164353 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:46.164425 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:46.164428 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1250,"child_traces":[["op",{"apply.queue_time_us":170,"apply.run_cpu_time_us":479,"apply.run_wall_time_us":480,"num_ops":1,"prepare.queue_time_us":48,"prepare.run_cpu_time_us":691,"prepare.run_wall_time_us":692,"raft.queue_time_us":238,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":237,"replication_time_us":14996}]]}
I20260618 13:59:47.006379 29026 catalog_manager.cc:2532] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_40fdb9a7e5ad4ff2bf879ec4e02a4b8d" } modify_external_catalogs: true
I20260618 13:59:47.006636 29026 catalog_manager.cc:2780] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_40fdb9a7e5ad4ff2bf879ec4e02a4b8d" } modify_external_catalogs: true
I20260618 13:59:47.006774 29019 catalog_manager.cc:3569] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33144:
table { table_name: "default.Table_38ca98ca393f4d549e1184885e55e64b" } new_table_name: "default.Table_1cf871317afd4e4ab9e417fdc8a30350" modify_external_catalogs: true
I20260618 13:59:47.007077 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dab5f1faf6bf46e2b7d9c3808511ef5a" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:47.007547 28618 leader_election.cc:343] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:47.007982 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 450c68da26404767b01c5e8da3f6286d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:46 UTC) from {username='slave'} at 127.0.0.1:38230
W20260618 13:59:47.008894 29024 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 60e02f51b8624a739fadbf4fafe90cb5 consensus state has no leader
I20260618 13:59:47.009021 30939 tablet_replica.cc:333] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:47.009212 30939 raft_consensus.cc:2243] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:47.009392 30939 raft_consensus.cc:2272] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:47.004240 29020 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_84be6926f4b64876b343577b85db3243 [id=af1859c644b44c1d831314d27a494f94] alter complete (version 1)
I20260618 13:59:47.009902 28401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dab5f1faf6bf46e2b7d9c3808511ef5a" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:47.010061 28401 raft_consensus.cc:2468] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:47.010291 31154 tablet.cc:1724] T f54910015987484e95b295352d859315 P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:47.010470 28617 leader_election.cc:304] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:47.014652 28651 tablet_service.cc:1558] Processing DeleteTablet for tablet 450c68da26404767b01c5e8da3f6286d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:46 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:47.015147 29566 tablet_replica.cc:333] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:47.015331 29566 raft_consensus.cc:2243] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:47.015587 29566 raft_consensus.cc:2272] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:47.016894 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 450c68da26404767b01c5e8da3f6286d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:46 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.017127 30950 tablet_replica.cc:333] T 450c68da26404767b01c5e8da3f6286d P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:47.017315 30950 raft_consensus.cc:2243] T 450c68da26404767b01c5e8da3f6286d P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:47.018836 31172 tablet.cc:1724] T f54910015987484e95b295352d859315 P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260618 13:59:47.020186 29024 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 153) took 3766 ms. Trace:
I20260618 13:59:47.020320 29024 rpcz_store.cc:276] 0618 13:59:43.253882 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:45.218363 (+1964481us) service_pool.cc:224] Handling call
0618 13:59:45.234638 (+ 16275us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb4686956 00007fcbab0e4146 00007fcbab0e3e75 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:45.235218 (+ 580us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:45.235472 (+ 254us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:45.235713 (+ 241us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:45.235806 (+ 93us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:45.277470 (+ 41664us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:47.020142 (+1742672us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:45.242540 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:45.242617 (+ 77us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:45.242621 (+ 4us) write_op.cc:435] Acquired schema lock
0618 13:59:45.242623 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:45.242687 (+ 64us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:45.242694 (+ 7us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:45.242697 (+ 3us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:45.242740 (+ 43us) tablet.cc:701] Row locks acquired
0618 13:59:45.242742 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:45.242765 (+ 23us) write_op.cc:270] Start()
0618 13:59:45.242787 (+ 22us) write_op.cc:276] Timestamp: P: 1781791185242721 usec, L: 0
0618 13:59:45.242790 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:45.242929 (+ 139us) log.cc:844] Serialized 405 byte log entry
0618 13:59:45.268611 (+ 25682us) op_driver.cc:464] REPLICATION: finished
0618 13:59:45.269763 (+ 1152us) write_op.cc:301] APPLY: starting
0618 13:59:45.269772 (+ 9us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:45.269802 (+ 30us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:45.269840 (+ 38us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:45.269870 (+ 30us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:45.269893 (+ 23us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:45.269902 (+ 9us) write_op.cc:312] APPLY: finished
0618 13:59:45.269958 (+ 56us) log.cc:844] Serialized 23 byte log entry
0618 13:59:45.270002 (+ 44us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:45.270016 (+ 14us) write_op.cc:454] Released schema lock
0618 13:59:45.270021 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":36634496,"child_traces":[["op",{"apply.queue_time_us":1123,"apply.run_cpu_time_us":323,"apply.run_wall_time_us":323,"num_ops":1,"prepare.queue_time_us":6279,"prepare.run_cpu_time_us":445,"prepare.run_wall_time_us":694,"replication_time_us":25805,"thread_start_us":17034,"threads_started":1}]]}
I20260618 13:59:47.020949 30939 ts_tablet_manager.cc:1916] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:47.021117 29024 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_37ac2499ad0a4b849e8fbe66130845ad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:47.021677 29024 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_37ac2499ad0a4b849e8fbe66130845ad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260618 13:59:47.030902 29019 catalog_manager.cc:5956] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4da3d6fa934047c09b3815c4bef511e1 consensus state has no leader
I20260618 13:59:47.031132 29019 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33144 (request call id 161) took 3003 ms. Trace:
I20260618 13:59:47.019068 30856 raft_consensus.cc:2804] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:47.031678 30856 raft_consensus.cc:493] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:47.031778 30856 raft_consensus.cc:3060] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:47.031241 29019 rpcz_store.cc:276] 0618 13:59:44.027390 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:47.006577 (+2979187us) service_pool.cc:224] Handling call
0618 13:59:47.006938 (+ 361us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:47.007176 (+ 238us) catalog_manager.cc:3750] Apply alter schema
0618 13:59:47.007349 (+ 173us) catalog_manager.cc:3782] Acquired catalog manager lock
0618 13:59:47.007429 (+ 80us) catalog_manager.cc:3940] Updating metadata on disk
0618 13:59:47.030577 (+ 23148us) catalog_manager.cc:3971] Committing alterations to in-memory state
0618 13:59:47.031066 (+ 489us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:47.008252 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:47.008346 (+ 94us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:47.008352 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:47.008354 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:47.008467 (+ 113us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:47.008481 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:47.008485 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:47.008547 (+ 62us) tablet.cc:701] Row locks acquired
0618 13:59:47.008549 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:47.008580 (+ 31us) write_op.cc:270] Start()
0618 13:59:47.008611 (+ 31us) write_op.cc:276] Timestamp: P: 1781791187008511 usec, L: 0
0618 13:59:47.008614 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:47.008788 (+ 174us) log.cc:844] Serialized 405 byte log entry
0618 13:59:47.029788 (+ 21000us) op_driver.cc:464] REPLICATION: finished
0618 13:59:47.029947 (+ 159us) write_op.cc:301] APPLY: starting
0618 13:59:47.029967 (+ 20us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:47.030017 (+ 50us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:47.030051 (+ 34us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:47.030110 (+ 59us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:47.030152 (+ 42us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:47.030162 (+ 10us) write_op.cc:312] APPLY: finished
0618 13:59:47.030247 (+ 85us) log.cc:844] Serialized 23 byte log entry
0618 13:59:47.030315 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:47.030333 (+ 18us) write_op.cc:454] Released schema lock
0618 13:59:47.030341 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:47.030455 (+ 114us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:47.030462 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":106,"apply.run_cpu_time_us":468,"apply.run_wall_time_us":466,"num_ops":1,"prepare.queue_time_us":270,"prepare.run_cpu_time_us":648,"prepare.run_wall_time_us":648,"raft.queue_time_us":10826,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":247,"replication_time_us":21140}]]}
I20260618 13:59:47.032287 29019 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_667adfea79864688a7ab9682517c21e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:47.032845 29019 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_667adfea79864688a7ab9682517c21e0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:47.032922 28797 tablet_bootstrap.cc:492] T 43853b8aba3444c3a21f6416f224e0d6 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:47.033046 28797 ts_tablet_manager.cc:1403] T 43853b8aba3444c3a21f6416f224e0d6 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.195s user 0.016s sys 0.016s
I20260618 13:59:47.033797 28797 raft_consensus.cc:359] T 43853b8aba3444c3a21f6416f224e0d6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.033933 28797 raft_consensus.cc:385] T 43853b8aba3444c3a21f6416f224e0d6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:47.033973 28797 raft_consensus.cc:740] T 43853b8aba3444c3a21f6416f224e0d6 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:47.034202 28797 consensus_queue.cc:260] T 43853b8aba3444c3a21f6416f224e0d6 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.034255 30856 raft_consensus.cc:515] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.024896 29566 ts_tablet_manager.cc:1916] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:47.034698 28797 ts_tablet_manager.cc:1434] T 43853b8aba3444c3a21f6416f224e0d6 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:47.034951 28797 tablet_bootstrap.cc:492] T 4da3d6fa934047c09b3815c4bef511e1 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:47.036908 28797 tablet_bootstrap.cc:654] T 4da3d6fa934047c09b3815c4bef511e1 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:47.043282 29566 ts_tablet_manager.cc:1929] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:47.043246 31154 tablet.cc:1724] T e196dd3c7fe14524a50152643ddbc2ad P 6fc808af0e884ed8836561665d8e0e7b: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260618 13:59:47.043483 29566 log.cc:1199] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/450c68da26404767b01c5e8da3f6286d
I20260618 13:59:47.043864 29566 ts_tablet_manager.cc:1950] T 450c68da26404767b01c5e8da3f6286d P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:47.044997 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dab5f1faf6bf46e2b7d9c3808511ef5a" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:47.045437 31055 raft_consensus.cc:493] T 5b51ac996e88453eb204e8b83f33863a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:47.045572 31055 raft_consensus.cc:515] T 5b51ac996e88453eb204e8b83f33863a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.046226 31055 leader_election.cc:290] T 5b51ac996e88453eb204e8b83f33863a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:47.046618 29007 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_3e7ca0884351432197632927d9c9db55 [id=5367efaf388c4ed78132d9752c341d5b] alter complete (version 2)
I20260618 13:59:47.044448 30856 leader_election.cc:290] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
W20260618 13:59:47.046766 28618 leader_election.cc:343] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:47.048734 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b51ac996e88453eb204e8b83f33863a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:47.048892 30986 tablet.cc:1724] T f54910015987484e95b295352d859315 P 97859748af2c41879ba2a668dda999d2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260618 13:59:47.049181 28346 leader_election.cc:343] T 5b51ac996e88453eb204e8b83f33863a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:47.049511 29018 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_c61ea74d5c6d4a718da2ab92f4a52397 [id=aec77306ae6d47ec91b286291c3ab664] alter complete (version 2)
I20260618 13:59:47.051023 31055 raft_consensus.cc:493] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:47.051172 31055 raft_consensus.cc:515] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.051787 31055 leader_election.cc:290] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:47.052161 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b602a473a94c05a617fb9478300c74" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:47.052364 28680 raft_consensus.cc:2468] T 00b602a473a94c05a617fb9478300c74 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 0.
I20260618 13:59:47.052737 28346 leader_election.cc:304] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:47.052971 31055 raft_consensus.cc:2804] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:47.053072 31055 raft_consensus.cc:493] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:47.053153 31055 raft_consensus.cc:3060] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:47.056321 31055 raft_consensus.cc:515] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.056949 31055 leader_election.cc:290] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:47.057451 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b602a473a94c05a617fb9478300c74" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257"
I20260618 13:59:47.057654 28680 raft_consensus.cc:3060] T 00b602a473a94c05a617fb9478300c74 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:47.060240 28680 raft_consensus.cc:2468] T 00b602a473a94c05a617fb9478300c74 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc808af0e884ed8836561665d8e0e7b in term 1.
I20260618 13:59:47.060668 28346 leader_election.cc:304] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters:
I20260618 13:59:47.060977 30864 raft_consensus.cc:2804] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:47.061102 30864 raft_consensus.cc:697] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [term 1 LEADER]: Becoming Leader. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Running, Role: LEADER
I20260618 13:59:47.061316 30864 consensus_queue.cc:237] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.064204 31056 consensus_queue.cc:1048] T 6903cb63c5b04408b7eaecf0c0231c42 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:47.067329 31056 consensus_queue.cc:1048] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:47.071854 31056 raft_consensus.cc:493] T 7eb0471903fd4654af395f7e4b68eb99 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:47.071988 31056 raft_consensus.cc:515] T 7eb0471903fd4654af395f7e4b68eb99 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.072629 31056 leader_election.cc:290] T 7eb0471903fd4654af395f7e4b68eb99 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:47.073338 28680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eb0471903fd4654af395f7e4b68eb99" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
W20260618 13:59:47.073822 28346 leader_election.cc:343] T 7eb0471903fd4654af395f7e4b68eb99 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:47.083755 28798 tablet_bootstrap.cc:492] T 51b91f053bcc42bb829bb8180e232c8e P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:47.083920 28798 ts_tablet_manager.cc:1403] T 51b91f053bcc42bb829bb8180e232c8e P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.266s user 0.003s sys 0.000s
I20260618 13:59:47.085866 31048 tablet_bootstrap.cc:492] T aa7faa3bb1b7474cb84e5f2a96af65e8 P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:47.086012 31048 ts_tablet_manager.cc:1403] T aa7faa3bb1b7474cb84e5f2a96af65e8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.253s user 0.003s sys 0.000s
I20260618 13:59:47.087168 31048 raft_consensus.cc:359] T aa7faa3bb1b7474cb84e5f2a96af65e8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.087388 31048 raft_consensus.cc:385] T aa7faa3bb1b7474cb84e5f2a96af65e8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:47.087462 31048 raft_consensus.cc:740] T aa7faa3bb1b7474cb84e5f2a96af65e8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc808af0e884ed8836561665d8e0e7b, State: Initialized, Role: FOLLOWER
I20260618 13:59:47.087728 31048 consensus_queue.cc:260] T aa7faa3bb1b7474cb84e5f2a96af65e8 P 6fc808af0e884ed8836561665d8e0e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.091892 30939 ts_tablet_manager.cc:1929] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:47.092067 30939 log.cc:1199] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/450c68da26404767b01c5e8da3f6286d
I20260618 13:59:47.092772 30939 ts_tablet_manager.cc:1950] T 450c68da26404767b01c5e8da3f6286d P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:47.099186 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]) successfully deleted
W20260618 13:59:47.100342 28617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 79343.5us
I20260618 13:59:47.102046 30856 raft_consensus.cc:493] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:47.102274 30856 raft_consensus.cc:515] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.102931 30856 leader_election.cc:290] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:47.103807 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dab5f1faf6bf46e2b7d9c3808511ef5a" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b"
I20260618 13:59:47.104156 28404 raft_consensus.cc:3060] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:47.107229 28404 raft_consensus.cc:2468] T dab5f1faf6bf46e2b7d9c3808511ef5a P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 1.
W20260618 13:59:47.116644 28988 catalog_manager.cc:4809] Async tablet task AlterTable RPC for tablet 60e02f51b8624a739fadbf4fafe90cb5 (table default.Table_bebdea167bb0486d89b2043d0c8351e5 [id=db6c4f85b83849128ab1b1447e7357be], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 60e02f51b8624a739fadbf4fafe90cb5 consensus state has no leader
I20260618 13:59:47.121943 30950 raft_consensus.cc:2272] T 450c68da26404767b01c5e8da3f6286d P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:47.123526 28543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b602a473a94c05a617fb9478300c74" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:47.123867 28545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eb0471903fd4654af395f7e4b68eb99" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:47.123991 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b602a473a94c05a617fb9478300c74" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:47.124814 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b51ac996e88453eb204e8b83f33863a" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:47.125563 30950 ts_tablet_manager.cc:1916] T 450c68da26404767b01c5e8da3f6286d P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 13:59:47.131204 28346 leader_election.cc:343] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:47.131538 28346 leader_election.cc:343] T 7eb0471903fd4654af395f7e4b68eb99 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:47.131629 28346 leader_election.cc:304] T 7eb0471903fd4654af395f7e4b68eb99 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
W20260618 13:59:47.132051 28346 leader_election.cc:343] T 00b602a473a94c05a617fb9478300c74 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:47.132185 30262 raft_consensus.cc:2749] T 7eb0471903fd4654af395f7e4b68eb99 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:47.133397 28544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9382a86a2f645b4857aa04e459209d9" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:47.134094 28618 consensus_peers.cc:597] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:47.134297 28618 leader_election.cc:343] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:47.134529 31075 raft_consensus.cc:493] T b2a6fabad00642528913e735929e295e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:47.134649 31075 raft_consensus.cc:515] T b2a6fabad00642528913e735929e295e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.135144 31075 leader_election.cc:290] T b2a6fabad00642528913e735929e295e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:47.136056 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2a6fabad00642528913e735929e295e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
W20260618 13:59:47.136328 28346 leader_election.cc:343] T 5b51ac996e88453eb204e8b83f33863a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:47.136423 28346 leader_election.cc:304] T 5b51ac996e88453eb204e8b83f33863a P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:47.136649 30262 raft_consensus.cc:2749] T 5b51ac996e88453eb204e8b83f33863a P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:47.138109 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2a6fabad00642528913e735929e295e" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:47.138865 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]) successfully deleted
I20260618 13:59:47.141125 30950 ts_tablet_manager.cc:1929] T 450c68da26404767b01c5e8da3f6286d P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 13:59:47.141307 30950 log.cc:1199] T 450c68da26404767b01c5e8da3f6286d P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/450c68da26404767b01c5e8da3f6286d
I20260618 13:59:47.141860 30950 ts_tablet_manager.cc:1950] T 450c68da26404767b01c5e8da3f6286d P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
I20260618 13:59:47.143627 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]) successfully deleted
I20260618 13:59:47.145733 28798 raft_consensus.cc:359] T 51b91f053bcc42bb829bb8180e232c8e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.145984 28798 raft_consensus.cc:385] T 51b91f053bcc42bb829bb8180e232c8e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:47.146078 28798 raft_consensus.cc:740] T 51b91f053bcc42bb829bb8180e232c8e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:47.146395 28798 consensus_queue.cc:260] T 51b91f053bcc42bb829bb8180e232c8e P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.154268 28458 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:47.163057 29011 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_0ac51b9af8d041798d81691aac77af3a [id=4cbcd521f63047a38f5a9b9def56df7f] alter complete (version 2)
I20260618 13:59:47.166158 29019 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 163) took 1810 ms. Trace:
I20260618 13:59:47.166292 29019 rpcz_store.cc:276] 0618 13:59:45.355538 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:47.032031 (+1676493us) service_pool.cc:224] Handling call
0618 13:59:47.032940 (+ 909us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:47.033525 (+ 585us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:47.165826 (+132301us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:47.165959 (+ 133us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:47.166080 (+ 121us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:47.062593 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:47.062728 (+ 135us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:47.062735 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:47.062738 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:47.062836 (+ 98us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:47.062847 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:47.062852 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:47.062938 (+ 86us) tablet.cc:701] Row locks acquired
0618 13:59:47.062940 (+ 2us) write_op.cc:260] PREPARE: finished
0618 13:59:47.062973 (+ 33us) write_op.cc:270] Start()
0618 13:59:47.063002 (+ 29us) write_op.cc:276] Timestamp: P: 1781791187062949 usec, L: 0
0618 13:59:47.063007 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:47.136945 (+ 73938us) log.cc:844] Serialized 433 byte log entry
0618 13:59:47.137161 (+ 216us) spinlock_profiling.cc:243] Waited 73.8 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fcbac75ef22 00007fcbadb40305 00007fcbadb3ee29 00007fcbadb47250 00007fcbadb4705d 0000000000aa681c 00007fcba07299ff 00007fcba072cd1b 00007fcba072cb98 0000000000aa681c
0618 13:59:47.165173 (+ 28012us) op_driver.cc:464] REPLICATION: finished
0618 13:59:47.165239 (+ 66us) write_op.cc:301] APPLY: starting
0618 13:59:47.165250 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:47.165282 (+ 32us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:47.165284 (+ 2us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:47.165369 (+ 85us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:47.165394 (+ 25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:47.165401 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:47.165471 (+ 70us) log.cc:844] Serialized 29 byte log entry
0618 13:59:47.165520 (+ 49us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:47.165534 (+ 14us) write_op.cc:454] Released schema lock
0618 13:59:47.165542 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":28,"apply.run_cpu_time_us":360,"apply.run_wall_time_us":358,"mutex_wait_us":28456,"num_ops":2,"prepare.queue_time_us":28537,"prepare.run_cpu_time_us":2026,"prepare.run_wall_time_us":74641,"raft.queue_time_us":1744,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":220,"replication_time_us":102142,"spinlock_wait_cycles":169645312}]]}
I20260618 13:59:47.168596 29007 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_55e47ed77544480a87dcb22d905c4418 [id=9102540068234d1993fa164035dabc6e] alter complete (version 1)
I20260618 13:59:47.176875 29024 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 162) took 1837 ms. Trace:
I20260618 13:59:47.176995 29019 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_6144d9f72234410fab68952869232a74"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:47.177016 29024 rpcz_store.cc:276] 0618 13:59:45.339369 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:47.020911 (+1681542us) service_pool.cc:224] Handling call
0618 13:59:47.021758 (+ 847us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:47.022215 (+ 457us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:47.176636 (+154421us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:47.176730 (+ 94us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:47.176820 (+ 90us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:47.026608 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:47.026718 (+ 110us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:47.026724 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:47.026728 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:47.026818 (+ 90us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:47.026829 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:47.026836 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:47.026914 (+ 78us) tablet.cc:701] Row locks acquired
0618 13:59:47.026918 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:47.026944 (+ 26us) write_op.cc:270] Start()
0618 13:59:47.026973 (+ 29us) write_op.cc:276] Timestamp: P: 1781791187026924 usec, L: 0
0618 13:59:47.026978 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:47.027168 (+ 190us) log.cc:844] Serialized 433 byte log entry
0618 13:59:47.135619 (+108451us) op_driver.cc:464] REPLICATION: finished
0618 13:59:47.135910 (+ 291us) write_op.cc:301] APPLY: starting
0618 13:59:47.135929 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:47.135976 (+ 47us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:47.136035 (+ 59us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:47.136235 (+ 200us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:47.136272 (+ 37us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:47.136279 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:47.136365 (+ 86us) log.cc:844] Serialized 29 byte log entry
0618 13:59:47.174295 (+ 37930us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:47.174310 (+ 15us) write_op.cc:454] Released schema lock
0618 13:59:47.174317 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:47.174386 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:47.174392 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":221,"apply.run_cpu_time_us":613,"apply.run_wall_time_us":38468,"num_ops":2,"prepare.queue_time_us":3723,"prepare.run_cpu_time_us":646,"prepare.run_wall_time_us":646,"raft.queue_time_us":140,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":3,"replication_time_us":108607}]]}
W20260618 13:59:47.177546 29019 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6144d9f72234410fab68952869232a74 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:47.177762 29024 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_172a75288db443a8b9ed3aaf8d0f79f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:47.178000 28798 ts_tablet_manager.cc:1434] T 51b91f053bcc42bb829bb8180e232c8e P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.094s user 0.000s sys 0.002s
W20260618 13:59:47.178258 29024 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_172a75288db443a8b9ed3aaf8d0f79f5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:47.179180 28798 tablet_bootstrap.cc:492] T d3ed5c1137434fcc80ed0ca56a323be7 P 97859748af2c41879ba2a668dda999d2: Bootstrap starting.
I20260618 13:59:47.181201 28798 tablet_bootstrap.cc:654] T d3ed5c1137434fcc80ed0ca56a323be7 P 97859748af2c41879ba2a668dda999d2: Neither blocks nor log segments found. Creating new log.
W20260618 13:59:47.183013 28617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.1:42675, user_credentials={real_user=slave}} blocked reactor thread for 65500.3us
I20260618 13:59:47.183706 31185 consensus_queue.cc:1048] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 13:59:47.184118 28618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 46080.2us
W20260618 13:59:47.189600 28346 consensus_peers.cc:597] T 60e02f51b8624a739fadbf4fafe90cb5 P 6fc808af0e884ed8836561665d8e0e7b -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:47.183766 30854 consensus_queue.cc:1048] T c200f34abf074684a0314c9024f04caf P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 13:59:47.191777 28797 tablet_bootstrap.cc:492] T 4da3d6fa934047c09b3815c4bef511e1 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:47.191965 28797 ts_tablet_manager.cc:1403] T 4da3d6fa934047c09b3815c4bef511e1 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.157s user 0.013s sys 0.034s
I20260618 13:59:47.192793 28797 raft_consensus.cc:359] T 4da3d6fa934047c09b3815c4bef511e1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.193092 28797 raft_consensus.cc:385] T 4da3d6fa934047c09b3815c4bef511e1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:47.193182 28797 raft_consensus.cc:740] T 4da3d6fa934047c09b3815c4bef511e1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:47.193392 28797 consensus_queue.cc:260] T 4da3d6fa934047c09b3815c4bef511e1 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.193981 28797 ts_tablet_manager.cc:1434] T 4da3d6fa934047c09b3815c4bef511e1 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260618 13:59:47.194362 28797 tablet_bootstrap.cc:492] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:47.197105 28797 tablet_bootstrap.cc:654] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:47.197876 31048 ts_tablet_manager.cc:1434] T aa7faa3bb1b7474cb84e5f2a96af65e8 P 6fc808af0e884ed8836561665d8e0e7b: Time spent starting tablet: real 0.112s user 0.003s sys 0.000s
I20260618 13:59:47.200019 31048 tablet_bootstrap.cc:492] T 55df96d305bf466b9fdea88a1816b2fa P 6fc808af0e884ed8836561665d8e0e7b: Bootstrap starting.
I20260618 13:59:47.202138 31048 tablet_bootstrap.cc:654] T 55df96d305bf466b9fdea88a1816b2fa P 6fc808af0e884ed8836561665d8e0e7b: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:47.183723 28617 leader_election.cc:304] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:47.203372 31188 raft_consensus.cc:2804] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 13:59:47.203511 31188 raft_consensus.cc:697] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Becoming Leader. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Running, Role: LEADER
I20260618 13:59:47.203934 31188 consensus_queue.cc:237] T dab5f1faf6bf46e2b7d9c3808511ef5a P d6a169b16ced46ca9cfbc631709bb257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.206959 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9382a86a2f645b4857aa04e459209d9" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:47.206984 29011 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_21426dbf385d47d2af5fc25e287ed0ed [id=0de5461053eb4517b15d6b91875fc04d] alter complete (version 1)
I20260618 13:59:47.207152 28404 raft_consensus.cc:2468] T c9382a86a2f645b4857aa04e459209d9 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:47.207888 28797 tablet_bootstrap.cc:492] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:47.208456 28797 ts_tablet_manager.cc:1403] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.004s
I20260618 13:59:47.209409 28797 raft_consensus.cc:359] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:47.210530 31075 raft_consensus.cc:493] T 52006500f117417b8a2c979d4aeb077c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:47.210711 31075 raft_consensus.cc:515] T 52006500f117417b8a2c979d4aeb077c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.211073 28797 raft_consensus.cc:385] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:47.211269 28797 raft_consensus.cc:740] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:47.211468 31075 leader_election.cc:290] T 52006500f117417b8a2c979d4aeb077c P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:47.212041 28404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52006500f117417b8a2c979d4aeb077c" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:47.212265 28404 raft_consensus.cc:2393] T 52006500f117417b8a2c979d4aeb077c P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate 6fc808af0e884ed8836561665d8e0e7b in this term.
I20260618 13:59:47.212886 28797 consensus_queue.cc:260] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } }
I20260618 13:59:47.213660 28797 ts_tablet_manager.cc:1434] T 3f9ec5f31f6c4d37b7c3ec585ed3211b P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260618 13:59:47.214382 28797 tablet_bootstrap.cc:492] T 7c40fe72e00a447582a35611de8ffa7c P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
I20260618 13:59:47.231160 29026 catalog_manager.cc:5983] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: Sending DeleteTablet for 3 replicas of tablet ba3d38d32c7c4721bad35f0da7a9ebe5
I20260618 13:59:47.232291 29026 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33144 (request call id 160) took 3232 ms. Trace:
I20260618 13:59:47.232524 29026 rpcz_store.cc:276] 0618 13:59:44.000124 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:47.006180 (+3006056us) service_pool.cc:224] Handling call
0618 13:59:47.006751 (+ 571us) catalog_manager.cc:2418] Looking up, locking, and authorizing table
0618 13:59:47.006920 (+ 169us) catalog_manager.cc:2840] Modifying in-memory table state
0618 13:59:47.006997 (+ 77us) catalog_manager.cc:2849] Locking tablets
0618 13:59:47.124842 (+117845us) catalog_manager.cc:2863] Removing table and tablets from system table
0618 13:59:47.203136 (+ 78294us) catalog_manager.cc:2881] Removing table from by-name map
0618 13:59:47.231036 (+ 27900us) catalog_manager.cc:2897] Committing in-memory state
0618 13:59:47.231104 (+ 68us) catalog_manager.cc:2901] Aborting table tasks
0618 13:59:47.232206 (+ 1102us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:47.164014 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:47.164181 (+ 167us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:47.164188 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:47.164193 (+ 5us) tablet.cc:662] Decoding operations
0618 13:59:47.164327 (+ 134us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:47.164338 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:47.164343 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:47.164491 (+ 148us) tablet.cc:701] Row locks acquired
0618 13:59:47.164495 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:47.164536 (+ 41us) write_op.cc:270] Start()
0618 13:59:47.164570 (+ 34us) write_op.cc:276] Timestamp: P: 1781791187164499 usec, L: 0
0618 13:59:47.164573 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:47.179474 (+ 14901us) log.cc:844] Serialized 762 byte log entry
0618 13:59:47.180171 (+ 697us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 000000000099c2aa 00007fcbac75ef22 00007fcbadb40305 00007fcbadb3ee29 00007fcbadb47250 00007fcbadb4705d 0000000000aa681c 00007fcba07299ff 00007fcba072cd1b 00007fcba072cb98 0000000000aa681c
0618 13:59:47.200549 (+ 20378us) op_driver.cc:464] REPLICATION: finished
0618 13:59:47.201489 (+ 940us) write_op.cc:301] APPLY: starting
0618 13:59:47.201512 (+ 23us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:47.201574 (+ 62us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:47.201578 (+ 4us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:47.201673 (+ 95us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:47.201720 (+ 47us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0618 13:59:47.201732 (+ 12us) write_op.cc:312] APPLY: finished
0618 13:59:47.201835 (+ 103us) log.cc:844] Serialized 29 byte log entry
0618 13:59:47.201907 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:47.201926 (+ 19us) write_op.cc:454] Released schema lock
0618 13:59:47.201932 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:47.202006 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:47.202010 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":857,"apply.run_cpu_time_us":508,"apply.run_wall_time_us":508,"num_ops":2,"prepare.queue_time_us":38489,"prepare.run_cpu_time_us":1408,"prepare.run_wall_time_us":16865,"raft.queue_time_us":119,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":324,"replication_time_us":35911,"spinlock_wait_cycles":33798016}]]}
I20260618 13:59:47.234347 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet ba3d38d32c7c4721bad35f0da7a9ebe5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:47 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.234639 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet ba3d38d32c7c4721bad35f0da7a9ebe5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:47 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:47.234709 29019 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 164) took 1869 ms. Trace:
I20260618 13:59:47.234786 29019 rpcz_store.cc:276] 0618 13:59:45.365207 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:47.167194 (+1801987us) service_pool.cc:224] Handling call
0618 13:59:47.176916 (+ 9722us) spinlock_profiling.cc:243] Waited 9.44 ms on lock 0x616000031954. stack: 00007fcba06e3496 00007fcba06e3250 00007fcb9f70e3ad 000000000099c914 000000000099c8dd 00007fcbafbc46bd 00007fcbafbc47b3 00007fcbac78cb21 00007fcbb44f195d 00007fcbb46846c3 00007fcbab0e07c6 00007fcbab0e04f5 00007fcba77ed88e 00007fcba77ec76e 00007fcba77f29a8 00007fcba77f330f
0618 13:59:47.177627 (+ 711us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:47.178065 (+ 438us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:47.203196 (+ 25131us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:47.234536 (+ 31340us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:47.234650 (+ 114us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:47.182026 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:47.182156 (+ 130us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:47.182163 (+ 7us) write_op.cc:435] Acquired schema lock
0618 13:59:47.182167 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:47.182259 (+ 92us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:47.182271 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:47.182277 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:47.182355 (+ 78us) tablet.cc:701] Row locks acquired
0618 13:59:47.182358 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:47.182391 (+ 33us) write_op.cc:270] Start()
0618 13:59:47.182446 (+ 55us) write_op.cc:276] Timestamp: P: 1781791187182354 usec, L: 0
0618 13:59:47.182451 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:47.182636 (+ 185us) log.cc:844] Serialized 433 byte log entry
0618 13:59:47.201224 (+ 18588us) op_driver.cc:464] REPLICATION: finished
0618 13:59:47.202605 (+ 1381us) write_op.cc:301] APPLY: starting
0618 13:59:47.202620 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:47.202660 (+ 40us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:47.202709 (+ 49us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:47.202839 (+ 130us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:47.202875 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:47.202885 (+ 10us) write_op.cc:312] APPLY: finished
0618 13:59:47.202950 (+ 65us) log.cc:844] Serialized 29 byte log entry
0618 13:59:47.203000 (+ 50us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:47.203012 (+ 12us) write_op.cc:454] Released schema lock
0618 13:59:47.203018 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":21705088,"child_traces":[["op",{"apply.queue_time_us":1338,"apply.run_cpu_time_us":465,"apply.run_wall_time_us":466,"num_ops":2,"prepare.queue_time_us":3417,"prepare.run_cpu_time_us":719,"prepare.run_wall_time_us":1098,"raft.queue_time_us":43,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":6,"replication_time_us":18751,"threads_started":1}]]}
I20260618 13:59:47.238345 28797 tablet_bootstrap.cc:654] T 7c40fe72e00a447582a35611de8ffa7c P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:47.242554 30950 tablet_replica.cc:333] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:47.243412 30950 raft_consensus.cc:2243] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:47.243709 30950 raft_consensus.cc:2272] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:47.244328 30950 ts_tablet_manager.cc:1916] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:47.245810 31188 raft_consensus.cc:493] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:47.244700 30552 raft_consensus.cc:493] T 0e6f3f8431cd49b5bf1306bc7609761d P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6fc808af0e884ed8836561665d8e0e7b)
I20260618 13:59:47.245954 31188 raft_consensus.cc:515] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.246066 30552 raft_consensus.cc:515] T 0e6f3f8431cd49b5bf1306bc7609761d P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.246518 31188 leader_election.cc:290] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:47.246984 30552 leader_election.cc:290] T 0e6f3f8431cd49b5bf1306bc7609761d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:47.248088 28401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e6f3f8431cd49b5bf1306bc7609761d" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:47.248303 31188 raft_consensus.cc:493] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:47.248446 31188 raft_consensus.cc:515] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.249135 31188 leader_election.cc:290] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:47.252875 30939 tablet_replica.cc:333] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b: stopping tablet replica
I20260618 13:59:47.253187 30939 raft_consensus.cc:2243] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:47.253396 30939 raft_consensus.cc:2272] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:47.254184 30939 ts_tablet_manager.cc:1916] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:47.262322 30939 ts_tablet_manager.cc:1929] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:47.262576 30939 log.cc:1199] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-0/wal/wals/ba3d38d32c7c4721bad35f0da7a9ebe5
I20260618 13:59:47.263059 30939 ts_tablet_manager.cc:1950] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 6fc808af0e884ed8836561665d8e0e7b: Deleting consensus metadata
I20260618 13:59:47.264351 28988 catalog_manager.cc:5027] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): tablet ba3d38d32c7c4721bad35f0da7a9ebe5 (table default.Table_40fdb9a7e5ad4ff2bf879ec4e02a4b8d [id=c1e1069f4a1640e2a3f86b302f1dcc7a]) successfully deleted
I20260618 13:59:47.265250 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 60b6216474b74a14af6abf977e312b77 with delete_type TABLET_DATA_DELETED (replaced by tablet f683917aac5945ec939664ab82c24530) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:47.265525 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet ed648dcf882340388155068d7a5a7869 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:44 UTC) from {username='slave'} at 127.0.0.1:38230
W20260618 13:59:47.266117 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 60b6216474b74a14af6abf977e312b77 because the tablet was not found. No further retry: Not found: Tablet not found: 60b6216474b74a14af6abf977e312b77
W20260618 13:59:47.266371 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet ed648dcf882340388155068d7a5a7869 because the tablet was not found. No further retry: Not found: Tablet not found: ed648dcf882340388155068d7a5a7869
I20260618 13:59:47.267335 28651 tablet_service.cc:1558] Processing DeleteTablet for tablet ba3d38d32c7c4721bad35f0da7a9ebe5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:47 UTC) from {username='slave'} at 127.0.0.1:43578
I20260618 13:59:47.267629 29566 tablet_replica.cc:333] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257: stopping tablet replica
I20260618 13:59:47.267858 29566 raft_consensus.cc:2243] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 LEADER]: Raft consensus shutting down.
I20260618 13:59:47.268191 29566 raft_consensus.cc:2272] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:47.268793 29566 ts_tablet_manager.cc:1916] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:47.276055 30950 ts_tablet_manager.cc:1929] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 97859748af2c41879ba2a668dda999d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:47.278739 30950 log.cc:1199] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 97859748af2c41879ba2a668dda999d2: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-1/wal/wals/ba3d38d32c7c4721bad35f0da7a9ebe5
I20260618 13:59:47.279143 30950 ts_tablet_manager.cc:1950] T ba3d38d32c7c4721bad35f0da7a9ebe5 P 97859748af2c41879ba2a668dda999d2: Deleting consensus metadata
W20260618 13:59:47.280040 28618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.100.2:40749, user_credentials={real_user=slave}} blocked reactor thread for 95569.8us
I20260618 13:59:47.280196 28989 catalog_manager.cc:5027] TS 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): tablet ba3d38d32c7c4721bad35f0da7a9ebe5 (table default.Table_40fdb9a7e5ad4ff2bf879ec4e02a4b8d [id=c1e1069f4a1640e2a3f86b302f1dcc7a]) successfully deleted
W20260618 13:59:47.280269 28618 consensus_peers.cc:597] T 7d43224c621a4774870d44f0237b514a P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260618 13:59:47.280582 28484 leader_election.cc:304] T b2a6fabad00642528913e735929e295e P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:47.282217 28538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44ad8cb18de42e795da5f229f49cdfe" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:47.283608 30552 raft_consensus.cc:2749] T b2a6fabad00642528913e735929e295e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:47.284790 28401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25463905e343470d941f89a53efbc2d1" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:47.284962 28401 raft_consensus.cc:2468] T 25463905e343470d941f89a53efbc2d1 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:47.286041 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44ad8cb18de42e795da5f229f49cdfe" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc808af0e884ed8836561665d8e0e7b" is_pre_election: true
I20260618 13:59:47.286369 28408 raft_consensus.cc:2468] T e44ad8cb18de42e795da5f229f49cdfe P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a169b16ced46ca9cfbc631709bb257 in term 0.
I20260618 13:59:47.286777 31202 raft_consensus.cc:493] T 66ada05b01b2427da182998a557529ca P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:47.286947 31202 raft_consensus.cc:515] T 66ada05b01b2427da182998a557529ca P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.287849 31202 leader_election.cc:290] T 66ada05b01b2427da182998a557529ca P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:47.293303 28682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52006500f117417b8a2c979d4aeb077c" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:47.293525 28682 raft_consensus.cc:2393] T 52006500f117417b8a2c979d4aeb077c P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97859748af2c41879ba2a668dda999d2 in current term 1: Already voted for candidate d6a169b16ced46ca9cfbc631709bb257 in this term.
I20260618 13:59:47.293633 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e6f3f8431cd49b5bf1306bc7609761d" candidate_uuid: "97859748af2c41879ba2a668dda999d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:47.294260 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ada05b01b2427da182998a557529ca" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
I20260618 13:59:47.297286 28484 leader_election.cc:304] T 52006500f117417b8a2c979d4aeb077c P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:47.297559 30552 raft_consensus.cc:3060] T 52006500f117417b8a2c979d4aeb077c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:47.297881 28484 leader_election.cc:304] T 0e6f3f8431cd49b5bf1306bc7609761d P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:47.298507 31075 raft_consensus.cc:2749] T 0e6f3f8431cd49b5bf1306bc7609761d P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 13:59:47.300851 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet c9ceb3af08fa4618b81cf6f89d7bd0df with delete_type TABLET_DATA_DELETED (replaced by tablet a1896e3b4a904fef8a741f63aa952251) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:47.301322 30552 raft_consensus.cc:2749] T 52006500f117417b8a2c979d4aeb077c P 97859748af2c41879ba2a668dda999d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260618 13:59:47.302183 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet c9ceb3af08fa4618b81cf6f89d7bd0df because the tablet was not found. No further retry: Not found: Tablet not found: c9ceb3af08fa4618b81cf6f89d7bd0df
I20260618 13:59:47.302282 28383 tablet_service.cc:1558] Processing DeleteTablet for tablet dbc88bf849a94cb9bd1c9c14aec53eb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:45 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:47.302564 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 91d014b46a0f448b929cbc9287ead0a7 with delete_type TABLET_DATA_DELETED (replaced by tablet aa7faa3bb1b7474cb84e5f2a96af65e8) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:47.303771 29026 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_cd047b08e5254b58b499fd04b91a8831"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:47.304622 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet dbc88bf849a94cb9bd1c9c14aec53eb1 because the tablet was not found. No further retry: Not found: Tablet not found: dbc88bf849a94cb9bd1c9c14aec53eb1
W20260618 13:59:47.304859 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 91d014b46a0f448b929cbc9287ead0a7 because the tablet was not found. No further retry: Not found: Tablet not found: 91d014b46a0f448b929cbc9287ead0a7
W20260618 13:59:47.304488 29026 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cd047b08e5254b58b499fd04b91a8831 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:47.306368 29019 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_a247a5ab24f14c9aa247cab4f105a664"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 13:59:47.306612 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25463905e343470d941f89a53efbc2d1" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:47.306859 29019 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a247a5ab24f14c9aa247cab4f105a664 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:47.310899 28798 tablet_bootstrap.cc:492] T d3ed5c1137434fcc80ed0ca56a323be7 P 97859748af2c41879ba2a668dda999d2: No bootstrap required, opened a new log
I20260618 13:59:47.311041 28798 ts_tablet_manager.cc:1403] T d3ed5c1137434fcc80ed0ca56a323be7 P 97859748af2c41879ba2a668dda999d2: Time spent bootstrapping tablet: real 0.132s user 0.002s sys 0.002s
I20260618 13:59:47.311851 28798 raft_consensus.cc:359] T d3ed5c1137434fcc80ed0ca56a323be7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.312017 28798 raft_consensus.cc:385] T d3ed5c1137434fcc80ed0ca56a323be7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:47.312077 28798 raft_consensus.cc:740] T d3ed5c1137434fcc80ed0ca56a323be7 P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97859748af2c41879ba2a668dda999d2, State: Initialized, Role: FOLLOWER
I20260618 13:59:47.312348 28798 consensus_queue.cc:260] T d3ed5c1137434fcc80ed0ca56a323be7 P 97859748af2c41879ba2a668dda999d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.313645 28617 leader_election.cc:304] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:47.314682 31187 raft_consensus.cc:2804] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:47.314782 31187 raft_consensus.cc:493] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:47.314869 31187 raft_consensus.cc:3060] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
W20260618 13:59:47.315763 28618 leader_election.cc:343] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:47.316118 28618 consensus_peers.cc:597] T 4a9a5f630654465fb416f53af1424e9b P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:47.316404 28618 leader_election.cc:343] T 66ada05b01b2427da182998a557529ca P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:47.316735 28618 consensus_peers.cc:597] T d7c6c17ba06543219259749d7d38832d P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:47.316908 28618 consensus_peers.cc:597] T 5cbfebfca436461ab63dfe469ae14b4c P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:47.317071 28618 leader_election.cc:343] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:47.317973 31187 raft_consensus.cc:515] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.318621 31187 leader_election.cc:290] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:47.318892 29024 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33144 (request call id 165) took 1953 ms. Trace:
I20260618 13:59:47.319252 30864 raft_consensus.cc:493] T aa7faa3bb1b7474cb84e5f2a96af65e8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:47.319411 30864 raft_consensus.cc:515] T aa7faa3bb1b7474cb84e5f2a96af65e8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.319026 29024 rpcz_store.cc:276] 0618 13:59:45.365293 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:47.177544 (+1812251us) service_pool.cc:224] Handling call
0618 13:59:47.178346 (+ 802us) catalog_manager.cc:2126] Acquired catalog manager lock
0618 13:59:47.178809 (+ 463us) catalog_manager.cc:2172] Created new table and tablet info
0618 13:59:47.318645 (+139836us) catalog_manager.cc:2232] Wrote table and tablets to system table
0618 13:59:47.318754 (+ 109us) catalog_manager.cc:2266] Inserted table and tablets into CatalogManager maps
0618 13:59:47.318851 (+ 97us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:47.183174 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:47.183311 (+ 137us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:47.183316 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:47.183319 (+ 3us) tablet.cc:662] Decoding operations
0618 13:59:47.183406 (+ 87us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:47.183428 (+ 22us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:47.183433 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0618 13:59:47.183510 (+ 77us) tablet.cc:701] Row locks acquired
0618 13:59:47.183514 (+ 4us) write_op.cc:260] PREPARE: finished
0618 13:59:47.183543 (+ 29us) write_op.cc:270] Start()
0618 13:59:47.183573 (+ 30us) write_op.cc:276] Timestamp: P: 1781791187183507 usec, L: 0
0618 13:59:47.183578 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:47.183731 (+ 153us) log.cc:844] Serialized 433 byte log entry
0618 13:59:47.303202 (+119471us) op_driver.cc:464] REPLICATION: finished
0618 13:59:47.315828 (+ 12626us) write_op.cc:301] APPLY: starting
0618 13:59:47.315848 (+ 20us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:47.316098 (+ 250us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:47.316159 (+ 61us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:47.317298 (+ 1139us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:47.317340 (+ 42us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0618 13:59:47.317348 (+ 8us) write_op.cc:312] APPLY: finished
0618 13:59:47.317475 (+ 127us) log.cc:844] Serialized 29 byte log entry
0618 13:59:47.317595 (+ 120us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:47.317616 (+ 21us) write_op.cc:454] Released schema lock
0618 13:59:47.317623 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":12527,"num_ops":2,"prepare.queue_time_us":3977,"prepare.run_cpu_time_us":646,"prepare.run_wall_time_us":991,"raft.queue_time_us":30,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":5,"replication_time_us":119591}]]}
I20260618 13:59:47.320013 30864 leader_election.cc:290] T aa7faa3bb1b7474cb84e5f2a96af65e8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503)
I20260618 13:59:47.319967 29024 catalog_manager.cc:2282] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33144:
name: "default.Table_3de6589e3bed4dca991a6a38d708df78"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 13:59:47.320552 29024 catalog_manager.cc:7076] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3de6589e3bed4dca991a6a38d708df78 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 13:59:47.320699 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa7faa3bb1b7474cb84e5f2a96af65e8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a169b16ced46ca9cfbc631709bb257" is_pre_election: true
I20260618 13:59:47.321095 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa7faa3bb1b7474cb84e5f2a96af65e8" candidate_uuid: "6fc808af0e884ed8836561665d8e0e7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2" is_pre_election: true
W20260618 13:59:47.321202 28346 leader_election.cc:343] T aa7faa3bb1b7474cb84e5f2a96af65e8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:47.321486 28346 leader_election.cc:343] T aa7faa3bb1b7474cb84e5f2a96af65e8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:47.321569 28346 leader_election.cc:304] T aa7faa3bb1b7474cb84e5f2a96af65e8 P 6fc808af0e884ed8836561665d8e0e7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b; no voters: 97859748af2c41879ba2a668dda999d2, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:47.321753 30864 raft_consensus.cc:2749] T aa7faa3bb1b7474cb84e5f2a96af65e8 P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:47.326960 28537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9382a86a2f645b4857aa04e459209d9" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:47.329268 28618 leader_election.cc:343] T c9382a86a2f645b4857aa04e459209d9 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:47.333429 28989 catalog_manager.cc:5100] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): alter failed for tablet 450c68da26404767b01c5e8da3f6286d (table default.Table_8f6f0bc5df454e3e9e307c07a4b86c53 [id=de842622055849ceac0bac11fbcba6e8]),no further retry: Not found: Tablet not found: 450c68da26404767b01c5e8da3f6286d
I20260618 13:59:47.334888 29566 ts_tablet_manager.cc:1929] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260618 13:59:47.335012 29566 log.cc:1199] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257: Deleting WAL directory at /tmp/dist-test-taskAN1Nhn/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1781791155674353-28048-0/minicluster-data/ts-2/wal/wals/ba3d38d32c7c4721bad35f0da7a9ebe5
I20260618 13:59:47.335414 29566 ts_tablet_manager.cc:1950] T ba3d38d32c7c4721bad35f0da7a9ebe5 P d6a169b16ced46ca9cfbc631709bb257: Deleting consensus metadata
I20260618 13:59:47.330696 29007 catalog_manager.cc:6241] T 00000000000000000000000000000000 P a950ab7cd66b4cb0b29592aa692c2dd0: default.Table_01e1e5741bb54f78996857f6b79fc5d7 [id=3d09bbb90ecb4103b2460f7aafa2db48] alter complete (version 2)
I20260618 13:59:47.342581 31172 tablet.cc:1724] T be00bfa9bc1a4b83b18bfad05ba1495e P d6a169b16ced46ca9cfbc631709bb257: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260618 13:59:47.348906 28517 tablet_service.cc:1511] Processing CreateTablet for tablet 8d620bc6be5249d8b0295ab379752c03 (DEFAULT_TABLE table=default.Table_667adfea79864688a7ab9682517c21e0 [id=9be76bd8c731465891d3c140486c0066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:47.349426 28517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d620bc6be5249d8b0295ab379752c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:47.349676 28617 leader_election.cc:304] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:47.349996 30856 raft_consensus.cc:2804] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:47.350052 28797 tablet_bootstrap.cc:492] T 7c40fe72e00a447582a35611de8ffa7c P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:47.350070 30856 raft_consensus.cc:493] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:47.350142 30856 raft_consensus.cc:3060] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:47.350159 28797 ts_tablet_manager.cc:1403] T 7c40fe72e00a447582a35611de8ffa7c P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.136s user 0.009s sys 0.013s
I20260618 13:59:47.351116 28797 raft_consensus.cc:359] T 7c40fe72e00a447582a35611de8ffa7c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.351483 28797 raft_consensus.cc:385] T 7c40fe72e00a447582a35611de8ffa7c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:47.351550 28797 raft_consensus.cc:740] T 7c40fe72e00a447582a35611de8ffa7c P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:47.351742 28797 consensus_queue.cc:260] T 7c40fe72e00a447582a35611de8ffa7c P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.351788 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet ed648dcf882340388155068d7a5a7869 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:44 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:47.352207 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 39b06ae7444a48dbbec9e096fd7e4052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:45 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:47.352941 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet dbc88bf849a94cb9bd1c9c14aec53eb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:45 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:47.353307 30856 raft_consensus.cc:515] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.354147 28375 tablet_service.cc:1558] Processing DeleteTablet for tablet 450c68da26404767b01c5e8da3f6286d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:46 UTC) from {username='slave'} at 127.0.0.1:38230
I20260618 13:59:47.354255 30856 leader_election.cc:290] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:47.352705 28797 ts_tablet_manager.cc:1434] T 7c40fe72e00a447582a35611de8ffa7c P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 13:59:47.354532 28798 ts_tablet_manager.cc:1434] T d3ed5c1137434fcc80ed0ca56a323be7 P 97859748af2c41879ba2a668dda999d2: Time spent starting tablet: real 0.043s user 0.001s sys 0.001s
I20260618 13:59:47.354655 28797 tablet_bootstrap.cc:492] T 8ac84641d0f1422193e42a42468789e1 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:47.354811 29011 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.100.1:48495 (request call id 156) took 1207 ms (client timeout 1000 ms). Trace:
I20260618 13:59:47.355261 30552 raft_consensus.cc:493] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 13:59:47.355635 30552 raft_consensus.cc:515] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
W20260618 13:59:47.354939 29011 rpcz_store.cc:269] 0618 13:59:46.146904 (+ 0us) service_pool.cc:167] Inserting onto call queue
0618 13:59:46.151538 (+ 4634us) service_pool.cc:224] Handling call
0618 13:59:47.354769 (+1203231us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0618 13:59:46.631831 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:46.632008 (+ 177us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:46.632019 (+ 11us) write_op.cc:435] Acquired schema lock
0618 13:59:46.632023 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:46.632238 (+ 215us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:46.632258 (+ 20us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:46.632266 (+ 8us) tablet.cc:685] Acquiring locks for 3 operations
0618 13:59:46.632484 (+ 218us) tablet.cc:701] Row locks acquired
0618 13:59:46.632489 (+ 5us) write_op.cc:260] PREPARE: finished
0618 13:59:46.632539 (+ 50us) write_op.cc:270] Start()
0618 13:59:46.632586 (+ 47us) write_op.cc:276] Timestamp: P: 1781791186632503 usec, L: 0
0618 13:59:46.632592 (+ 6us) op_driver.cc:348] REPLICATION: starting
0618 13:59:46.632825 (+ 233us) log.cc:844] Serialized 1284 byte log entry
0618 13:59:46.636602 (+ 3777us) op_driver.cc:464] REPLICATION: finished
0618 13:59:46.636786 (+ 184us) write_op.cc:301] APPLY: starting
0618 13:59:46.636808 (+ 22us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:46.636877 (+ 69us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:46.636913 (+ 36us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:46.637020 (+ 107us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:46.637070 (+ 50us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0618 13:59:46.637081 (+ 11us) write_op.cc:312] APPLY: finished
0618 13:59:46.637184 (+ 103us) log.cc:844] Serialized 35 byte log entry
0618 13:59:46.637264 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:46.637294 (+ 30us) write_op.cc:454] Released schema lock
0618 13:59:46.637304 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:46.637396 (+ 92us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:46.637406 (+ 10us) write_op.cc:454] Released schema lock
Related trace 'op':
0618 13:59:47.034549 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:47.034644 (+ 95us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:47.034649 (+ 5us) write_op.cc:435] Acquired schema lock
0618 13:59:47.034651 (+ 2us) tablet.cc:662] Decoding operations
0618 13:59:47.034734 (+ 83us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:47.034743 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:47.034747 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:47.034805 (+ 58us) tablet.cc:701] Row locks acquired
0618 13:59:47.034808 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:47.034837 (+ 29us) write_op.cc:270] Start()
0618 13:59:47.034861 (+ 24us) write_op.cc:276] Timestamp: P: 1781791187034818 usec, L: 0
0618 13:59:47.034864 (+ 3us) op_driver.cc:348] REPLICATION: starting
0618 13:59:47.035014 (+ 150us) log.cc:844] Serialized 356 byte log entry
0618 13:59:47.159603 (+124589us) op_driver.cc:464] REPLICATION: finished
0618 13:59:47.162551 (+ 2948us) write_op.cc:301] APPLY: starting
0618 13:59:47.162568 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:47.162606 (+ 38us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:47.162638 (+ 32us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:47.162691 (+ 53us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:47.162719 (+ 28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:47.162726 (+ 7us) write_op.cc:312] APPLY: finished
0618 13:59:47.162802 (+ 76us) log.cc:844] Serialized 23 byte log entry
0618 13:59:47.162852 (+ 50us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:47.162869 (+ 17us) write_op.cc:454] Released schema lock
0618 13:59:47.162876 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0618 13:59:47.180914 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0618 13:59:47.181032 (+ 118us) write_op.cc:432] Acquiring schema lock in shared mode
0618 13:59:47.181038 (+ 6us) write_op.cc:435] Acquired schema lock
0618 13:59:47.181042 (+ 4us) tablet.cc:662] Decoding operations
0618 13:59:47.181175 (+ 133us) write_op.cc:620] Acquiring the partition lock for write op
0618 13:59:47.181189 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0618 13:59:47.181195 (+ 6us) tablet.cc:685] Acquiring locks for 1 operations
0618 13:59:47.181262 (+ 67us) tablet.cc:701] Row locks acquired
0618 13:59:47.181265 (+ 3us) write_op.cc:260] PREPARE: finished
0618 13:59:47.181305 (+ 40us) write_op.cc:270] Start()
0618 13:59:47.181337 (+ 32us) write_op.cc:276] Timestamp: P: 1781791187181266 usec, L: 0
0618 13:59:47.181342 (+ 5us) op_driver.cc:348] REPLICATION: starting
0618 13:59:47.181530 (+ 188us) log.cc:844] Serialized 356 byte log entry
0618 13:59:47.201147 (+ 19617us) op_driver.cc:464] REPLICATION: finished
0618 13:59:47.202089 (+ 942us) write_op.cc:301] APPLY: starting
0618 13:59:47.202104 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0618 13:59:47.202146 (+ 42us) tablet.cc:1370] finished BulkCheckPresence
0618 13:59:47.202197 (+ 51us) tablet.cc:1372] starting ApplyRowOperation cycle
0618 13:59:47.202248 (+ 51us) tablet.cc:1383] finished ApplyRowOperation cycle
0618 13:59:47.202285 (+ 37us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0618 13:59:47.202297 (+ 12us) write_op.cc:312] APPLY: finished
0618 13:59:47.202368 (+ 71us) log.cc:844] Serialized 23 byte log entry
0618 13:59:47.202441 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:47.202462 (+ 21us) write_op.cc:454] Released schema lock
0618 13:59:47.202472 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0618 13:59:47.202536 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0618 13:59:47.202540 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":36721,"reported_tablets":147,"child_traces":[["op",{"apply.queue_time_us":111,"apply.run_cpu_time_us":598,"apply.run_wall_time_us":598,"num_ops":3,"prepare.queue_time_us":54,"prepare.run_cpu_time_us":1126,"prepare.run_wall_time_us":1126,"raft.queue_time_us":875,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":241,"replication_time_us":3968}],["op",{"apply.queue_time_us":2876,"apply.run_cpu_time_us":396,"apply.run_wall_time_us":396,"num_ops":1,"prepare.queue_time_us":1587,"prepare.run_cpu_time_us":544,"prepare.run_wall_time_us":545,"raft.queue_time_us":163,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":8,"replication_time_us":124707}],["op",{"apply.queue_time_us":888,"apply.run_cpu_time_us":440,"apply.run_wall_time_us":440,"num_ops":1,"prepare.queue_time_us":15370,"prepare.run_cpu_time_us":714,"prepare.run_wall_time_us":1080,"raft.queue_time_us":37,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":19770}]]}
I20260618 13:59:47.356288 29011 master_service.cc:960] ReplaceTablet: received request to replace tablet 6903cb63c5b04408b7eaecf0c0231c42 from {username='slave'} at 127.0.0.1:33144
I20260618 13:59:47.356803 30552 leader_election.cc:290] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:47.359192 28617 leader_election.cc:304] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257; no voters: 97859748af2c41879ba2a668dda999d2
I20260618 13:59:47.359470 30856 raft_consensus.cc:2804] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 13:59:47.359587 30856 raft_consensus.cc:493] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 13:59:47.359702 30856 raft_consensus.cc:3060] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:47.360711 28797 tablet_bootstrap.cc:654] T 8ac84641d0f1422193e42a42468789e1 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:47.362365 30856 raft_consensus.cc:515] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.362963 30856 leader_election.cc:290] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Requested vote from peers 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749), 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675)
I20260618 13:59:47.364089 28651 tablet_service.cc:1511] Processing CreateTablet for tablet 8d620bc6be5249d8b0295ab379752c03 (DEFAULT_TABLE table=default.Table_667adfea79864688a7ab9682517c21e0 [id=9be76bd8c731465891d3c140486c0066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:47.364445 28651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d620bc6be5249d8b0295ab379752c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:47.364732 28797 tablet_bootstrap.cc:492] T 8ac84641d0f1422193e42a42468789e1 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:47.364843 28797 ts_tablet_manager.cc:1403] T 8ac84641d0f1422193e42a42468789e1 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
W20260618 13:59:47.364809 28483 leader_election.cc:343] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): Not found: Tablet not found: c9ceb3af08fa4618b81cf6f89d7bd0df
I20260618 13:59:47.365403 28797 raft_consensus.cc:359] T 8ac84641d0f1422193e42a42468789e1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.365535 28797 raft_consensus.cc:385] T 8ac84641d0f1422193e42a42468789e1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:47.365577 28797 raft_consensus.cc:740] T 8ac84641d0f1422193e42a42468789e1 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:47.365736 28797 consensus_queue.cc:260] T 8ac84641d0f1422193e42a42468789e1 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.369758 28545 raft_consensus.cc:3060] T b2a6fabad00642528913e735929e295e P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:47.369984 28540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25463905e343470d941f89a53efbc2d1" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
I20260618 13:59:47.370381 28540 raft_consensus.cc:3060] T c5836982752b4209b72a372c1dfeb71c P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 13:59:47.357147 28989 catalog_manager.cc:5027] TS d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): tablet ba3d38d32c7c4721bad35f0da7a9ebe5 (table default.Table_40fdb9a7e5ad4ff2bf879ec4e02a4b8d [id=c1e1069f4a1640e2a3f86b302f1dcc7a]) successfully deleted
W20260618 13:59:47.359124 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet ed648dcf882340388155068d7a5a7869 because the tablet was not found. No further retry: Not found: Tablet not found: ed648dcf882340388155068d7a5a7869
I20260618 13:59:47.376106 28542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44ad8cb18de42e795da5f229f49cdfe" candidate_uuid: "d6a169b16ced46ca9cfbc631709bb257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97859748af2c41879ba2a668dda999d2"
W20260618 13:59:47.376646 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 39b06ae7444a48dbbec9e096fd7e4052 because the tablet was not found. No further retry: Not found: Tablet not found: 39b06ae7444a48dbbec9e096fd7e4052
W20260618 13:59:47.376871 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet dbc88bf849a94cb9bd1c9c14aec53eb1 because the tablet was not found. No further retry: Not found: Tablet not found: dbc88bf849a94cb9bd1c9c14aec53eb1
W20260618 13:59:47.377072 28988 catalog_manager.cc:4994] TS 6fc808af0e884ed8836561665d8e0e7b (127.27.100.1:42675): delete failed for tablet 450c68da26404767b01c5e8da3f6286d because the tablet was not found. No further retry: Not found: Tablet not found: 450c68da26404767b01c5e8da3f6286d
I20260618 13:59:47.377188 28375 tablet_service.cc:1511] Processing CreateTablet for tablet 8d620bc6be5249d8b0295ab379752c03 (DEFAULT_TABLE table=default.Table_667adfea79864688a7ab9682517c21e0 [id=9be76bd8c731465891d3c140486c0066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:47.377661 28375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d620bc6be5249d8b0295ab379752c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:47.377789 28384 tablet_service.cc:1511] Processing CreateTablet for tablet a0e564adb88a4eb280ebab3ffe922199 (DEFAULT_TABLE table=default.Table_37ac2499ad0a4b849e8fbe66130845ad [id=c0e2b9ee33ad4557a6289b774ece11c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:47.378104 28384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0e564adb88a4eb280ebab3ffe922199. 1 dirs total, 0 dirs full, 0 dirs failed
W20260618 13:59:47.379623 28618 leader_election.cc:343] T 25463905e343470d941f89a53efbc2d1 P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
W20260618 13:59:47.379786 28618 leader_election.cc:343] T e44ad8cb18de42e795da5f229f49cdfe P d6a169b16ced46ca9cfbc631709bb257 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260618 13:59:47.380262 28797 ts_tablet_manager.cc:1434] T 8ac84641d0f1422193e42a42468789e1 P d6a169b16ced46ca9cfbc631709bb257: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260618 13:59:47.380515 28797 tablet_bootstrap.cc:492] T 7c96a195e54440d4854e4462d178b468 P d6a169b16ced46ca9cfbc631709bb257: Bootstrap starting.
W20260618 13:59:47.382381 28618 consensus_peers.cc:597] T daface32f2a94a49a2db0729dfba32c2 P d6a169b16ced46ca9cfbc631709bb257 -> Peer 97859748af2c41879ba2a668dda999d2 (127.27.100.2:40749): Couldn't send request to peer 97859748af2c41879ba2a668dda999d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260618 13:59:47.382680 28484 leader_election.cc:343] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6a169b16ced46ca9cfbc631709bb257 (127.27.100.3:35503): Not found: Tablet not found: c9ceb3af08fa4618b81cf6f89d7bd0df
I20260618 13:59:47.382784 28484 leader_election.cc:304] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97859748af2c41879ba2a668dda999d2; no voters: 6fc808af0e884ed8836561665d8e0e7b, d6a169b16ced46ca9cfbc631709bb257
I20260618 13:59:47.382979 31075 raft_consensus.cc:2749] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 13:59:47.383955 28517 tablet_service.cc:1558] Processing DeleteTablet for tablet 91d014b46a0f448b929cbc9287ead0a7 with delete_type TABLET_DATA_DELETED (replaced by tablet aa7faa3bb1b7474cb84e5f2a96af65e8) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.384217 28527 tablet_service.cc:1558] Processing DeleteTablet for tablet c9ceb3af08fa4618b81cf6f89d7bd0df with delete_type TABLET_DATA_DELETED (replaced by tablet a1896e3b4a904fef8a741f63aa952251) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.384931 28512 tablet_service.cc:1558] Processing DeleteTablet for tablet 60b6216474b74a14af6abf977e312b77 with delete_type TABLET_DATA_DELETED (replaced by tablet f683917aac5945ec939664ab82c24530) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.384373 28519 tablet_service.cc:1558] Processing DeleteTablet for tablet 2128dbec33db453ba51070767a15b21e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.384539 28511 tablet_service.cc:1558] Processing DeleteTablet for tablet ed648dcf882340388155068d7a5a7869 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:44 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.385095 28529 tablet_service.cc:1558] Processing DeleteTablet for tablet f45556a59c424ca2bc30956566374c89 with delete_type TABLET_DATA_DELETED (replaced by tablet 6903cb63c5b04408b7eaecf0c0231c42) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.385118 28518 tablet_service.cc:1558] Processing DeleteTablet for tablet 39b06ae7444a48dbbec9e096fd7e4052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:45 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.385303 28525 tablet_service.cc:1558] Processing DeleteTablet for tablet ba2d6894bf5649fc8cfae7e033c432b9 with delete_type TABLET_DATA_DELETED (replaced by tablet a6ba4976bb6e4c2da3a563ccad4eb2b0) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.385456 28523 tablet_service.cc:1558] Processing DeleteTablet for tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.385565 28797 tablet_bootstrap.cc:654] T 7c96a195e54440d4854e4462d178b468 P d6a169b16ced46ca9cfbc631709bb257: Neither blocks nor log segments found. Creating new log.
I20260618 13:59:47.385603 28526 tablet_service.cc:1558] Processing DeleteTablet for tablet dbc88bf849a94cb9bd1c9c14aec53eb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:45 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.385740 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet 72c04f55890648f0a8184f3783243f7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.385921 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet 98446f40214c4ac984fcfe98310005c5 with delete_type TABLET_DATA_DELETED (replaced by tablet 240c1f7ac1ee4fcb9e48dc3d4bbc4dd3) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.386224 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet c01c8463055e4526b3c75190066f30b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.387269 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet 450c68da26404767b01c5e8da3f6286d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:46 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.387722 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet e3d8616846c649b6a969b971db0d03db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:32 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.388044 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet fea86899d2c047d997cebe80551e83c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:41 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.388419 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet 8fea86ca38a741238a59053151087f69 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f9ec5f31f6c4d37b7c3ec585ed3211b) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.388553 30950 tablet_replica.cc:333] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2: stopping tablet replica
I20260618 13:59:47.388731 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet 6c4a31a4f202473d97b17b5b5dfdb615 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:35 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.389139 28521 tablet_service.cc:1558] Processing DeleteTablet for tablet 962bf304a285412fb2a529db5b994e9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:34 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.389319 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet f2b21dcf74fa4264947ca43cc0026e76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 13:59:40 UTC) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.389813 28528 tablet_service.cc:1511] Processing CreateTablet for tablet a0e564adb88a4eb280ebab3ffe922199 (DEFAULT_TABLE table=default.Table_37ac2499ad0a4b849e8fbe66130845ad [id=c0e2b9ee33ad4557a6289b774ece11c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 13:59:47.390164 28528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0e564adb88a4eb280ebab3ffe922199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 13:59:47.388739 30950 raft_consensus.cc:2243] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260618 13:59:47.392344 30950 raft_consensus.cc:2272] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260618 13:59:47.392856 30950 ts_tablet_manager.cc:1916] T c9ceb3af08fa4618b81cf6f89d7bd0df P 97859748af2c41879ba2a668dda999d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 13:59:47.400874 28528 tablet_service.cc:1558] Processing DeleteTablet for tablet d4667fe48f0f4eff9c6c8baabdab58d4 with delete_type TABLET_DATA_DELETED (replaced by tablet a662bb37fb7d4df49e61e524808b67a8) from {username='slave'} at 127.0.0.1:57282
I20260618 13:59:47.401607 28797 tablet_bootstrap.cc:492] T 7c96a195e54440d4854e4462d178b468 P d6a169b16ced46ca9cfbc631709bb257: No bootstrap required, opened a new log
I20260618 13:59:47.401772 28797 ts_tablet_manager.cc:1403] T 7c96a195e54440d4854e4462d178b468 P d6a169b16ced46ca9cfbc631709bb257: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.004s
I20260618 13:59:47.402714 28797 raft_consensus.cc:359] T 7c96a195e54440d4854e4462d178b468 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.402944 28797 raft_consensus.cc:385] T 7c96a195e54440d4854e4462d178b468 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 13:59:47.402994 28797 raft_consensus.cc:740] T 7c96a195e54440d4854e4462d178b468 P d6a169b16ced46ca9cfbc631709bb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a169b16ced46ca9cfbc631709bb257, State: Initialized, Role: FOLLOWER
I20260618 13:59:47.403208 28797 consensus_queue.cc:260] T 7c96a195e54440d4854e4462d178b468 P d6a169b16ced46ca9cfbc631709bb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97859748af2c41879ba2a668dda999d2" member_type: VOTER last_known_addr { host: "127.27.100.2" port: 40749 } } peers { permanent_uuid: "d6a169b16ced46ca9cfbc631709bb257" member_type: VOTER last_known_addr { host: "127.27.100.3" port: 35503 } } peers { permanent_uuid: "6fc808af0e884ed8836561665d8e0e7b" member_type: VOTER last_known_addr { host: "127.27.100.1" port: 42675 } }
I20260618 13:59:47.404174 28730 heartbeater.cc:499] Master 127.27.100.61:45967 was elected leader, sending a full tablet report...
I20260618 13:59:47.406678 31048 tablet_bootstrap.cc:492] T 55df96d305bf466b9fdea88a1816b2fa P 6fc808af0e884ed8836561665d8e0e7b: No bootstrap required, opened a new log
I20260618 13:59:47.406859 31048 ts_tablet_manager.cc:1403] T 55df96d305bf466b9fdea88a1816b2fa P 6fc808af0e884ed8836561665d8e0e7b: Time spent bootstrapping tablet: real 0.209s user 0.004s sys 0.000s
I20260618 13:59:47.407781 31048 raft_consensus.cc:359] T 55df96d305bf466b9fdea88a1816b2fa P 6fc808af0e884ed8836561665d8e0e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "